JP2017049977A - Information addition method and server apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To extract only an image about a specific object from a large amount of images not inclusive of an intention of a photographer, and to generate a set of large-scale image data.SOLUTION: An information addition method is configured to: acquire an image transmitted, via a network, from any of at least one or more terminals having a function of capturing an image or acquiring an image from another device via the network; calculate a probability that a prescribed capturing object is included in the acquired image; and when the probability is higher than a first threshold, add information indicative of the prescribed capturing object to the image, and when the probability is lower than a second threshold, not add the information indicative of the prescribed capturing object to the image, and when the probability is equal to or higher than the second threshold, and equal to or lower than the first threshold, transmit the image and request reception information for receiving addition of the information with respect to the image to any of the at least one or more terminals via the network.SELECTED DRAWING: Figure 4

Description

  The present disclosure relates to a method for adding information to an image or the like.

  In recent years, the accuracy of image recognition technology called Deep Learning has been rapidly improved. This deep learning has a feature that it automatically learns a feature amount used for recognition unlike the conventional case. On the other hand, a large-scale learning image data set is required to construct a deep learning recognition system.

  In Deep Learning, when you want to recognize any specific object included in an image (including any living organisms and structures such as animals and stairs), a large-scale image data set that collects images showing that specific object Is used for learning. In order to improve the recognition accuracy, it is known that it is effective to collect a wide variety of images showing the recognition target in a comprehensive manner. Even if it is not limited to using for deep learning, there is an increasing need to collect various images that show a specific object to be photographed, such as when operating a service that provides specific images for research purposes. .

  Patent Documents 1 and 2 each disclose a technique for collecting a large amount of images.

Japanese Patent No. 4297193 Japanese Patent Laid-Open No. 10-267671

  However, in Patent Document 1 or Patent Document 2, only an image related to a specific object is extracted from a large amount of images that do not include the photographer's intention, and a large-scale image data set is generated (or an image related to a specific object). The method of assigning tag information) has not been studied.

  In order to solve the above-described problem, an information providing method according to the present disclosure is a server information providing method, in which a processor of the server acquires an image from a function of capturing an image or another device via a network. An image transmitted via a network from any one or more terminals having a function is acquired, and a probability that a predetermined photographing object is included is calculated with respect to the acquired image, and the probability is the first If it is above a threshold, information indicating the predetermined object to be photographed is given to the image, and if the probability is below a second threshold, the predetermined object to be photographed is given to the image. When the information indicating an object is not given and the probability is equal to or higher than the second threshold and equal to or lower than the first threshold, the image and request acceptance information for accepting information addition to the image; 1 or more Transmission via the network to one of the terminals.

  These general or specific aspects may be realized by a system, a method, an integrated circuit, a computer program, or a recording medium, and are realized by any combination of the system, method, integrated circuit, computer program, and recording medium. May be.

  According to the present disclosure, it is possible to provide information comprehensively and efficiently to various captured images that do not reflect a person's intention while suppressing the human cost of the entire system.

FIG. 1 is a diagram illustrating an example of an outline of a system. FIG. 2 is a diagram illustrating an example of the configuration of the imaging device and the server. FIG. 3 is a diagram illustrating an example of a functional configuration of the calculation unit. FIG. 4 is a diagram illustrating an example of processing performed between the imaging device and the server. FIG. 5 is a flowchart showing an example of processing relating to probability derivation and processing discrimination. FIG. 6 is a flowchart illustrating an example of processing relating to probability derivation and processing discrimination. FIG. 7 is a diagram illustrating a display example of the terminal when performing determination input.

(Knowledge that became the basis of the invention)
In collecting a large amount of images of a specific object, there is a method as in Patent Document 1 in which a photographer requests a photographer to acquire an image including a specific object and causes the photographer to capture the image. In this technique, the photographing client issues a specific instruction to the photographer regarding what kind of object is necessary, that is, the object to be photographed. However, in this case, unless otherwise specified by the client, the photographer has no obstruction in front of the object to be photographed, the object is centered, the object to be photographed is not blurred, Try to take a beautiful image. That is, at the time when the photographer performs photographing, the photographed image includes the intention of the photographer, and there are many similar images in which the photographed object is clearly photographed. The image data set used for learning improves image recognition accuracy when there are few similar images in the same image data set (that is, when there are many dissimilar images and various images). On the other hand, there is a possibility that the image data set collected by the method of Patent Document 1 includes many similar images.

  Patent Document 2 discloses a method of automatically acquiring an image by installing a camera on a moving object such as a car. This makes it possible to acquire a large amount of images that do not include human intentions. However, in this case, the acquired image does not always include the photographing object. For this reason, the photographer needs to determine whether or not the captured object is included in all acquired images. In Patent Document 2, position information is used for discrimination of an object included in an acquired image. In other words, if the method of Patent Document 2 is used, if the location always changes, such as “road” or “landmark”, the image is automatically included in the image data set related to “road” or “landmark”. There is a possibility that a judgment can be made. However, for example, when the object to be imaged is such that the position information is constantly changing, such as “car”, it is impossible to determine whether to include the image data set of “car” based on the imaged position. Moreover, there is a limit in automatically determining what is reflected in an image acquired based on position information. For example, if the sky is photographed even at the same position, there is a possibility that a desired photographing object such as “road” is not included. Therefore, in the end, when the method of Patent Document 2 is used, it is necessary to determine whether or not a person includes an object to be photographed for all acquired images.

  In order to solve these problems, the information providing method according to the present disclosure is an information providing method of a server, and the processor of the server has one or more functions of capturing an image or acquiring an image from another device. An image transmitted from any of the terminals via a network is acquired, and a probability that a predetermined photographing object is included is calculated for the acquired image, and the probability is above a first threshold value In the case, information indicating the predetermined photographing object is given to the image, and when the probability is lower than a second threshold, information indicating the predetermined photographing object is given to the image. If the probability is not less than the second threshold and not more than the first threshold, the image and the request acceptance information for accepting information addition to the image are not less than the first threshold. Any of the devices Transmitting via the network.

  This makes it possible to acquire a large amount of images, and those that can easily determine whether or not the object to be photographed are included in a short time by a machine, and those that are difficult to determine are image collection / A tagging system is built. Therefore, it is possible to give a comprehensive and large amount of information to various captured images that do not reflect the intention of the person while suppressing the human cost of the entire system.

  Further, the request acceptance information may include a command for displaying an instruction screen for accepting selection from the terminal as to whether or not the photographing object is included in the image.

  Thereby, the user holding the terminal can easily give information to the image.

  Further, a selection result indicating whether or not the photographing object is included in the image received based on the instruction screen is acquired from the terminal, and the selection result is a selection result indicating that the photographing object is included. If there is, information indicating the predetermined photographing object is given to the image, and if the selection result is a selection result indicating that the photographing object is not included, the predetermined photographing object is included in the image. It is good also as not giving the information which shows the imaging | photography subject.

  As a result, it is possible to determine whether or not information is added (or not added) appropriately even for an image that is difficult to be determined by the server.

  In addition, for the calculation of the probability, an arithmetic unit for calculating a probability that a predetermined photographing object held in advance by the server is included may be used.

  Further, the computing unit may be a computing unit constructed by deep learning, Hog (Histogram of Oriented Gradient), SVM (Support Vector Machine), or a combination thereof.

  Further, by applying information using the information providing method to a plurality of images, a plurality of images to which information is added is accumulated, and a plurality of images to which the information is added are used. The calculator may be updated.

  As a result, as the number of images to which information is added by the information providing method increases, the accuracy with respect to information providing by the information providing method is improved.

  Further, when calculating the probability using the arithmetic unit constructed using an image to which the information of less than a predetermined number is given, and using an image to which the information of a predetermined number or more is given. The first threshold value and the second threshold value may be changed when the probability is calculated using the constructed arithmetic unit.

  Further, when calculating the probability using the computing unit constructed using an image provided with less than a predetermined number of information, an image provided with more than a predetermined number of information is used. The first threshold may be set higher and the second threshold may be set lower than when the probability is calculated using the constructed arithmetic unit.

  As a result, when the number of images to which information is added by this information providing method is less than the predetermined number, since the computing unit is not constructed, the first threshold value is set higher and the second threshold value is set lower. It is possible to increase the number of pieces of information added by the terminal user (photographer).

  In the acquisition, the server acquires the device ID of the terminal from the terminal together with the image. In the transmission, the server specifies the device ID acquired together with the image. The image and the request acceptance information may be transmitted to the terminal having the specified device ID.

  Thereby, information can be given to the image by the same person (photographer) who acquired the image, so that it is possible to efficiently determine whether each image corresponds to a predetermined object.

  In the acquisition, the device ID of the terminal is acquired from the terminal together with the image. In the transmission of the image and the request reception information, the device ID is acquired together with the image, and the device ID is specified. Of these, the image and the request acceptance information may be transmitted to a terminal different from a terminal having a device ID different from the specified device ID.

  As a result, information can be assigned to the image by a person other than the person who acquired the image, so that an objective determination can be made with respect to the information addition.

  Furthermore, after processing whether to give information to the image by the information giving method, information for providing a reward to the terminal is transmitted, and the reward is related to the probability that the image is related to the image. When the probability is greater than or equal to 2 and less than or equal to the first threshold, the probability is higher than the first threshold and the probability is lower than the second threshold. Also good.

  As a result, the reward is high when there are a large number of images in which the probability relating to the image is equal to or higher than the second threshold and equal to or lower than the first threshold. Can grasp the image requested by

  In addition, before acquiring the image, a selection related to the predetermined photographing object may be received from the terminal.

  Each of the embodiments described below shows a specific example of the present disclosure. Numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements. In all the embodiments, the contents can be combined.

(Embodiment 1)
(Overview of the system)
FIG. 1 is a diagram showing an outline of an image collection system in the present embodiment. Hereinafter, an outline of the image collection system will be described with reference to FIG.

  The image collection system includes a photographing device 100 owned by the photographer 1 and a server device 200 owned by the photograph requester 2.

  The imaging device 100 is a mobile terminal capable of communication, input, image capturing and display, such as smart glasses. The server device 200 is a cloud server that collects a large amount of information from various imaging devices and processes the information.

  In addition, although the imaging requester 2 is expressed as possessing the server apparatus 200, the place where the server apparatus 200 is placed and the management entity are not particularly limited. It suffices if the client 2 manages the server device 200 directly or indirectly.

  The imaging device 100 and the server device 200 are connected via a network such as the Internet, and can exchange information with each other.

  According to the present disclosure, first, the photographer 1 acquires a large amount of images by the photographing device 100 and sends the images to the server device 200 owned by the photographing client 2 via the network. Further, the client 2 determines whether or not a desired subject is included in the image by processing in the server device 200. Here, the desired subject of the client 2 is referred to as a subject to be photographed. The object to be photographed is not limited to an object, and includes people and living things. In addition, the photograph requester 2 sends an image that does not know whether or not the photographing target is included to the photographing device 100 owned by the photographer 1. Further, the photographer 1 sends to the server apparatus 200 owned by the photographing requester whether or not the photographing object is included in the image. A detailed processing flow will be described later.

  Note that it is desirable that there are a plurality of photographers 1. That is, the server device 200 may be connected to a plurality of imaging devices 100.

  The imaging client 2 is an individual or group that needs a large amount of images including a specific imaging object for the purpose of learning or evaluating the image classifier. When collecting images for learning or evaluation of the image discriminator, obtaining a wide variety of images leads to an improvement in accuracy.

  Therefore, for example, when the image photographed by the photographer 1 is a rare image, the photograph requester 2 provides a service that gives the photographer 1 a high reward for acquiring the rare image.

  A rare image is, for example, an image that includes a shooting object, although it does not resemble an image that includes a shooting object already owned by the client 2. Whether or not the image is rare is determined by deriving the probability that the image capturing object is included in the image in the server apparatus 200 owned by the image capturing client 2. If it is determined that the derived probability is low and the image includes an imaging target, it is determined that the image is a rare image. In addition, when position information can be acquired in the mobile terminal, it may be determined whether or not the image is rare using the information. As a result, the photographer 1 actively collects images including photographing objects in more various places and scenes in order to obtain more rewards. Details of the processing such as probability derivation will be described later.

(Configuration of each device)
The configuration of the imaging device 100 and the server device 200 in the present embodiment will be described with reference to FIGS. Various configurations of the photographing apparatus 100 and the server apparatus 200 can be considered, but this time, description will be made using the simplest example.

  The photographing device 100 can be any movable photographing / photographing device such as a wearable device such as a smart glass, a portable device such as a digital camera / smartphone, or a camera installed in a moving means such as a car or a motorcycle. Includes communication terminals.

  The imaging device 100 includes an imaging unit 110, an imaging device communication unit 120, a display unit 130, and an input unit 140, as shown in FIG.

  The imaging unit 110 acquires an image from the surrounding environment automatically or manually. The imaging unit 110 includes any device that is mounted on the imaging device 100 and has an imaging function. For example, when the photographing apparatus 100 is a smartphone, it is an imaging device (CCD image sensor or CMOS image sensor) mounted on the smartphone.

  The imaging device communication unit 120 performs communication between the imaging device 100 and the server device 200. The imaging device communication unit 120 may adopt a general-purpose communication module, and is not particularly limited. The communication method used when communicating with the server device 200 is not particularly limited, for example, 3G / LTE / WiFi.

  The display unit 130 displays a captured image and an operation screen. The display unit 130 may be a general-purpose display device, and is not particularly limited.

  The input unit 140 receives a determination result by the photographer as to whether or not the photographed image displayed on the display unit 130 includes a photographing object. The input unit 140 includes all interfaces such as a touch panel and voice input in addition to buttons.

  The photographing apparatus 100 may not include all the above-described configurations (the photographing unit 110, the photographing device communication unit 120, the display unit 130, and the input unit 140). Some configurations may be included in another device. For example, a configuration in which the photographing unit 110 is performed by a single camera and the photographing device communication unit 120 is performed by a communication device such as a smartphone is possible. As the photographing unit 110, any camera that can communicate with the photographing device communication unit, such as a wearable camera and an in-vehicle camera, can be adopted in addition to a normal camera. In the case where the photographing unit 110 is capable of always photographing, such as a smart glass or a wearable camera, a large amount of images that are not arbitrary can be obtained by automatically obtaining images. The image acquired by the camera does not need to have high image quality.

  The server device 200 includes a control processing unit 210 and a server device communication unit 220 as shown in FIG.

  The control processing unit 210 will be described in detail with reference to FIG. The control processing unit 210 includes a probability derivation unit 211, a process determination unit 212, and an image management unit 213.

  The probability deriving unit 211 calculates a probability that a certain image captured by the imaging device 100 includes the object to be imaged.

  The process determining unit 212 determines subsequent processing of the image according to the probability that the image obtained by the probability deriving unit 211 includes the shooting target.

  The image management unit 213 stores the image acquired by the imaging unit 110 and determined to include the imaging target by the input unit 140 or the processing determination unit 212. The image management unit 213 discards the image acquired by the imaging unit 110 and determined not to include the imaging target by the input unit 140 or the processing determination unit 212.

  The reward deriving unit 214 derives a reward obtained by the photographer 1 using the probability that the image obtained by the probability deriving unit 211 includes the photographing object for the image including the photographing object. Note that the reward deriving unit 214 may derive the reward by using the position information together with the position information if it is attached to the image.

  The server device communication unit 220 performs communication between the inside of the server device 200 and between the imaging device 100 and the server device 200. A communication module similar to that of the imaging device communication unit 120 may be used.

  The server device 200 may not include all the above-described configurations (the control processing unit 210 and the server device communication unit 220). The control processing unit 210 may not include all the functions described above (probability derivation means 211, process determination means 212, image management means 213, and reward derivation means 214). Some configurations and functions may be included in another device. A device including a part of the configuration and functions may be owned by a third party other than the client 2.

(Process flow)
Next, the flow of processing in the present embodiment will be described with reference to FIG.

  FIG. 4 is a diagram illustrating an example of processing performed between the imaging device and the server.

  The photographing apparatus 100 performs photographing of an image and manual determination of whether or not a photographing object is included in the image, accepts selection of request information, step S304 that performs photographing processing, step S306 that performs photographing processing, and step 312 described later. Step S313 for displaying the transmitted image, and step S314 for receiving a determination input as to whether or not the displayed image includes the object to be photographed are performed.

  The server device 200 derives the probability of whether an object is included in the image and determines the processing of the image, assigns a photographer ID, and derives the probability of whether the object is included in the image. Step S308, step S309 for determining subsequent processing based on the probability derived in step S308, step S310 for saving the image (first image) acquired in step S307, which will be described later, and discarding the first image Step S311 to perform, Step S316 for storing the image (second image) determined to include the object to be photographed in Step S314, Step S317 for discarding the second image, and Step S318 for calculating the reward for the image are performed. .

  Data communication is performed between the photographing apparatus 100 and the server device 200, and step S301 for performing user registration, step S303 for transmitting request information, step S305 for transmitting selected request information, and an acquired image Step S307 for sending, step S312 for sending the discrimination image, step S315 for sending the discrimination result of the discrimination image, and step S319 for sending the reward based on the calculation in step S318 are performed.

  The details of the above steps will be described below.

  In step S <b> 301, the photographer 1 performs user registration with the server apparatus 200 using the photographing apparatus 100. That is, in step S <b> 301, the photographing apparatus 100 transmits registration information received from the photographer 1 to the server device 200. The registration information at this time may include, for example, information such as used terminal information, residence, knowledge range, age, and gender in addition to the name.

  Next, in step S <b> 302, the server apparatus 200 allocates a photographer ID that is an ID of a user (photographer 1) who has registered with the photographing apparatus 100.

  Alternatively, in step S <b> 302, the server apparatus 200 may allocate a device ID that is an ID of the photographing device 100 used by a user who has registered with the photographing device 100.

  Alternatively, for example, when the user uses the imaging device 100 and a terminal device (not shown) that communicates with the server device 200, the terminal device acquires an image captured by the imaging device 100, and the acquired image is stored in the server. In the case of transmission to the device 200, in step S302, the photographer ID and device IDs of a plurality of devices (in this example, the photographing device 100 and the terminal device) used by the photographer of this ID may be associated and allocated.

  Next, in step S <b> 303, the server apparatus 200 transmits (introduces) request information to the imaging device 100. After step S302, the server apparatus 200 may perform selection of a request to introduce to the imaging device 100 using the information registered in step S302. The request information transmitted in step S303 may include a plurality of request information. The plurality of pieces of request information may include a request from the photographing client 2 and a request from other business operators. The request information specifically includes information for designating an object to be photographed, such as “dog image” or “car image”.

  Next, in step S <b> 304, the photographing apparatus 100 accepts selection of request information from the photographer 1. The photographer 1 determines request information to be executed by selecting desired request information from the request information received by the photographing apparatus 100 in step S303. The request information is selected by an input unit (for example, a button / touch panel or voice input) of the photographing apparatus 100. That is, the object to be photographed is determined by the processing in step S304.

  Next, in step S <b> 305, the imaging device 100 transmits the request information selected in step S <b> 304 and the photographer ID of the imaging device 100 to the server device 200.

  Next, in step S306, the imaging unit 110 of the imaging device 100 acquires an image from the surrounding environment of the photographer. Acquisition of images in the imaging unit 110 is performed automatically or manually by the photographer 1. In the case of automatic, in response to an instruction from a crowdsourcing company designated in advance, an image shown in the photographing unit 110 at that time is acquired about once every several tens of seconds or several minutes. In the case of automatic, an image shown in the photographing unit 110 may be acquired according to the position information of the photographing device 100. For example, if there are two points X and Y, and the number of photos at point X is less than that at point Y, the image acquisition frequency at point X is set higher than the image acquisition frequency at point Y. Many images can be acquired with a small number. The acquired image is stored in a recording unit (not shown) of the photographing apparatus 100. The storage at this time is temporary, and may be deleted after transmission in step S307 to be described next.

  Next, in step S307, the imaging device communication unit 120 of the imaging device 100 transmits the image acquired by the imaging unit 110 to the server device communication unit 220. That is, in step S307, the server device communication unit 220 receives an image transmitted to the server device 200 by the photographing device communication unit 120. At this time, the photographer ID or device ID, position information at the time of photographing, terminal information used, etc. may be transmitted and received together with the image. At this time, the timing at which the imaging device communication unit 120 transmits an image to the server device 200 is not particularly limited. Each time an image is captured in step S306, the images may be transmitted one by one, or may be transmitted when a predetermined number of images have been acquired, or may be specified by the photographer 1 via the input unit 140. You may transmit at a timing.

  Next, in step S308, the probability deriving unit 211 derives the probability that the image received in step S307 includes the photographing object determined in step S304. The probability is derived, for example, by inputting the image received in step S307 to a computing unit for identifying the subject to be photographed that has been constructed in advance by deep learning included in the control processing unit 210. Here, the arithmetic unit may be an arithmetic unit constructed by an image recognition method using not only deep learning but also Hog, SVM, or a combination thereof.

  In step S309, the process determining unit 212 determines subsequent processes based on the probability derived by the probability deriving unit 211. Details of the processing in step S309 will be described with reference to FIG. It is assumed that probabilities A and B corresponding to threshold values necessary for determination are set before executing step S309. Probabilities A and B are A> B. In step S309, the process determination unit 212 receives the probability derived by the probability deriving unit 211 as an input. It is determined whether the input probability is higher than A (step S3901). If the probability derived by the probability deriving unit 211 is higher than A (Yes in step S3901), it is determined that the image used for deriving the input probability includes a photographing object, and step S310 is executed. If the probability derived by the probability deriving means 211 is equal to or less than A (No in step S3901), step S3902 is executed. In step S3902, it is determined whether the probability derived by the probability deriving unit 211 is lower than B. If the probability derived by the probability deriving unit 211 is lower than B (Yes in step S3902), it is determined that the image used for deriving the input probability does not include the imaging target, and step S311 is executed.

  If the probability derived by the probability deriving means 211 is B or higher (No in step S3902), step S312 is executed.

  In step S309, since the process shown in FIG. 5 is performed, if the probability derived by the probability deriving unit 211 is larger than A, it is determined that the image used for deriving the input probability includes the imaging target. Step S310 is executed. If the probability derived by the probability deriving unit 211 is equal to or lower than A and smaller than B, it is determined that the image used for deriving this probability does not include the photographing object, and step S311 is executed. If the probability derived by the probability deriving means 211 is A or less and B or more, step S312 is executed. The probabilities A and B necessary for the determination may be changed as appropriate according to the burden on the photographer and the required image level. In addition, a probability calculator (also referred to as a computing unit) including a photographing object used in the probability deriving unit 211 is newly created using a collected image (for example, an image to which a tag of the photographing object is attached). (Or rebuilt) and updated. The values of the probabilities A and B may be changed in accordance with the update of the computer in the probability deriving means 211. The computer in the probability deriving unit 211 becomes more accurate by updating it using a newly acquired image. When the probability can be derived with higher accuracy, it is possible to reduce the number of images to be manually tagged by setting the probabilities A and B appropriately. A specific example of the process determination step S309 will be described with reference to FIG. FIG. 6 is a specific example of step S309 when the photographing object is a dog. Three images acquired by the photographing apparatus 100 are referred to as an image X, an image Y, and an image Z. In this case, in step S308, the probability deriving unit 211 derives the probability that the image includes a dog. Assume that the probability that an image includes a dog is 95% for image X, 50% for image Y, and 5% for image Z in step S308. The probabilities A and B for discrimination are set to 85% and 20%, respectively. In this case, in step S309, the process determination unit 212 receives the probability derived by the probability deriving unit 211 as an input. At this time, since the probability that the dog is included in the image X is higher than 85%, it is determined that the dog is a dog, and the process proceeds to step S310 to be stored in the server device 200. Next, since the probability that a dog is included in image Y is 85% or less and 20% or more, it is determined that the machine does not know whether it is a dog or not, and the flow proceeds to step S312 to perform manual determination. Finally, since the probability that a dog is included in the image Z is lower than 20%, it is determined that the dog is not a dog, and the process proceeds to step S311 to discard it.

  Returning to FIG. 4, the process flow will be described. In step S310, the image management unit 213 adds a tag of the shooting target to the image determined to include the shooting target (determined to be higher than the probability A) in step S309, and the server apparatus 200 It is stored in a memory (not shown) provided. The memory is a writable recording medium such as a hard disk or a writable semiconductor memory.

  In step S <b> 311, the image management unit 213 discards the image determined not to include the imaging target (determined to be lower than the probability B) in step S <b> 309 without adding the tag of the imaging target.

  In step S312, the server apparatus communication unit 220 captures the request acceptance information for accepting an image determined to be equal to or less than the probability A and the probability B or more in step S309 and tagging corresponding to the information addition to the image. Transmit to device 100. That is, in step S312, the imaging device communication unit 120 receives an image transmitted to the imaging device 100 by the server device communication unit 220.

  For example, the image determined in step S309 to be less than or equal to the probability A and greater than or equal to the probability B is an image received from the imaging device 100 via the network in step S307.

  For example, if the photographer ID or device ID is received together with the image in step S307, and if this image is determined to be less than probability A and greater than probability B in step S309, In S307, the photographer ID or device ID received together with this image is specified. Then, this image and request acceptance information are transmitted to the device corresponding to the specified photographer ID or the device corresponding to the device ID. For example, consider a case where a plurality of devices are registered in association with the photographer ID of the photographer 1 in the registration of the user (in this example, the photographer 1) in step S302. In this case, in step S312, the image and the request reception information may be transmitted to a device (for example, a smartphone or a tablet-type mobile terminal) that can easily perform tagging work associated with the photographer ID. In this way, those that perform the tagging work can more easily perform the tagging work.

  In step S312, the image and request acceptance information may be sent to a device having a photographer ID different from the photographer ID received together with this image in step S307. In this way, this image may be sent to a person (photographer) different from the person who sent the image to the server device 200 (image acquirer). Sending an image to a more handy photographer, separate from the image acquirer, makes tagging faster. Further, by sending an image to a plurality of photographers, it becomes possible to perform tagging with higher accuracy. When the image is sent back to the acquirer of the image, it is possible to send the photographer ID together with the image in step S307 to the same image acquirer.

  In step S312, the device ID received together with this image in step S307 may be specified, and the image and request acceptance information may be sent to a device having a device ID different from the specified device ID.

  In step S313, the display unit 130 receives the request reception information received by the imaging device communication unit 120, and includes an instruction screen that includes the received image and receives a selection as to whether or not the imaging target is included in the image from the terminal. indicate. The instruction screen includes, for example, an icon image (for example, an icon image corresponding to “Yes” and “No”) for selecting whether or not a shooting target is included.

  In step S <b> 314, the input unit 140 receives a determination result by the photographer as to whether or not the image displayed on the display unit 130 includes a photographing object.

  FIG. 7 shows an example of an interface when displaying an image in step S313. In the process determination step S309, an image determined to be unable to be determined by the machine whether or not an object to be photographed is included (an image transmitted in step S312 and determined to be less than or equal to probability A and greater than or equal to probability B) is: In step S313, the image is displayed on the display unit 130 as in the example of FIG. The photographer 1 determines whether or not the object to be imaged is included in the image displayed in step S313 in step S314, and inputs the determination result from the input unit 140.

  In step S <b> 315, the imaging device communication unit 120 transmits the determination result by the photographer to the server apparatus 200 as to whether or not the image input by the input unit 140 includes the imaging target. That is, in step S315, the server apparatus communication unit 220 receives a determination result by the photographer as to whether or not the image input by the input unit 140 includes a shooting target. When the determination result that the image includes the photographing target is received, step S316 is performed. If the determination result that the image does not include the object to be photographed is received, step S317 is performed.

  In step S316, the image management unit 213 adds a tag of the shooting target to the image determined to include the shooting target in step S314, and stores it in a memory (not shown) provided in the server device 200. As a result of the determination that the image includes the object to be photographed, if an image tagged with the object to be photographed is received in step S315, the image management unit 213 stores the image in the memory in step S316. Good.

  In step S317, the image management unit 213 discards the image determined not to include the shooting target object in step S314.

  In step S318, the reward deriving unit 214 derives a reward using the probability obtained in step S310 that the imaged object is included. Here, for example, the reward is derived higher as the number of images transmitted in step S312 increases (that is, as the number of images determined to be lower than probability A and higher than probability B in step S309).

  In step S319, the server device 200 sends a reward to the photographer to the photographing apparatus 100. The reward includes everything that benefits the photographer, such as money, net money, game points, discount tickets, lottery tickets, and the like.

  Note that the derivation of the probability in step S308 is, for example, as described above using an arithmetic unit for identifying an imaging target that is built in advance by deep learning included in the control processing unit 210. More specifically, for example, learning by deep learning is performed using a data set of a plurality of learning images prepared in advance for recognition of a photographing object, and an arithmetic unit for identifying the photographing object is provided in advance. Built.

  For example, a plurality of images stored in step S310 and step S316 may be added to the learning image data set to perform learning, and an arithmetic unit for identifying a photographing object may be updated (or reconstructed). . By performing such an update, the accuracy (or reliability) of the probability derived using the computing unit becomes higher.

  The timing at which the computing unit is updated may be, for example, the timing at which the image is stored in step S310 and step S316, or may be performed before executing step S308.

  For example, in the case of deriving the probability using an arithmetic unit constructed using less than a predetermined number of images and the case of deriving the probability using an arithmetic unit constructed using a predetermined number of images or more, the steps In 309, the values of probabilities A and B necessary for discrimination may be changed.

  For example, when the probability is derived using an arithmetic unit constructed using less than a predetermined number of images, the value of the probability A to be set is a probability obtained using an arithmetic unit constructed using a predetermined number of images or more. May be set to a value higher than the value of the probability A to be set.

  In addition, for example, when the probability is derived using an arithmetic unit constructed using less than a predetermined number of images, the probability B to be set is set using an arithmetic unit constructed using a predetermined number of images or more. When the probability is derived, it may be set to a value lower than the value of the probability B to be set.

  For example, the predetermined number of sheets may be determined according to an imaging target to be recognized, or may be determined as appropriate by an administrator who manages the server device 200.

  As described above, a large amount of images can be acquired, and those that can be easily determined whether or not the object to be photographed are included in a short time with a recognizer. -A tagging system is established. In the conventional patent, when collecting a large amount of images including an object to be photographed, a large number of images including the photographer's intention are collected, and the image is biased. In addition, when a large amount of images that do not include the photographer's intention are collected, a specific method for giving information as to whether or not a subject to be photographed is included in the large amount of images has not been studied. Therefore, it is necessary to manually add information on whether or not the object to be photographed is included in all the collected images, which requires a lot of labor. However, according to the present disclosure, a comprehensive and large amount of various captured images that include a desired object to be captured and do not include a person's intention are collected and intended information (shooting It is possible to efficiently give information on whether or not it is an object while suppressing the human cost and load of the entire system. As a result, a large amount of various image data sets (image data sets that are not biased to similar images) to which information indicating that the object is a desired object to be photographed are used as learning images, resulting in higher recognition performance. An image recognizer can be constructed. Therefore, for example, an image recognition system using a technique such as Deep Learning can be constructed more efficiently and accurately than in the past.

  As above, the processing of the present disclosure has been described in the embodiment, but there is no particular limitation on the subject or apparatus in which each processing is performed. It may be processed by a processor or the like (described below) incorporated in a specific device located locally. Further, it may be processed by a cloud server or the like arranged at a location different from the local device. Moreover, you may share each process demonstrated by this indication by coordinating information between a local apparatus and a cloud server. Embodiments of the present disclosure will be described below.

  (1) Specifically, the above apparatus is a computer system including a microprocessor, ROM, RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or hard disk unit. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  (2) A part or all of the constituent elements constituting the above-described apparatus may be constituted by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  (3) A part or all of the constituent elements constituting the above-described device may be constituted by an IC card that can be attached to and detached from each device or a single module. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  (4) The present disclosure may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  (5) In addition, the present disclosure provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD ( It may be recorded on a Blu-ray (registered trademark) disc, a semiconductor memory, or the like. The digital signal may be recorded on these recording media.

  In addition, the present disclosure may transmit the computer program or the digital signal via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present disclosure may be a computer system including a microprocessor and a memory, the memory storing the computer program, and the microprocessor operating according to the computer program.

  In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like and executed by another independent computer system. You may do that.

  (6) The above embodiment and its modifications may be combined.

  The present disclosure relates to an object recognition method and system capable of accurately recognizing what an object included in an image is from an image, and imaging of a digital camera, a movie, a surveillance camera, an in-vehicle camera, a wearable camera, and the like. Useful as a device.

DESCRIPTION OF SYMBOLS 100 Image pick-up apparatus 110 Image pick-up part 120 Image pick-up apparatus communication part 130 Display part 140 Input part 200 Server apparatus 210 Control processing part 211 Probability deriving means 212 Process determination means 213 Image management means 214 Reward deriving means

Claims (15)

A server information providing method, wherein the server processor includes:
Obtaining an image transmitted via a network from one or more terminals having a function of capturing an image or a function of obtaining an image from another device;
For the acquired image, calculate the probability that a predetermined shooting object is included,
If the probability is above a first threshold, give the image information indicating the predetermined object to be photographed,
When the probability is lower than the second threshold, information indicating the predetermined photographing object is not given to the image,
If the probability is greater than or equal to the second threshold and less than or equal to the first threshold, the image and request acceptance information for accepting information assignment to the image may be any of the one or more terminals. Send over the network,
Information grant method. The request acceptance information includes a command for displaying an instruction screen for accepting selection from the terminal as to whether or not the photographing object is included in the image.
The information providing method according to claim 1. Further, a selection result as to whether or not the photographing object is included in the image received based on the instruction screen is acquired from the terminal,
When the selection result is a selection result indicating that the photographing object is included, information indicating the predetermined photographing object is given to the image,
When the selection result is a selection result indicating that the shooting object is not included, information indicating the predetermined shooting object is not given to the image.
The information providing method according to claim 2. For the calculation of the probability, an arithmetic unit for calculating the probability that the server includes a predetermined photographing object held in advance is used.
The information providing method according to claim 1. The computing unit is a computing unit constructed by Deep Learning, Hog (Histogram of Oriented Gradient), SVM (Support Vector Machine), or a combination thereof.
The information providing method according to claim 4. By giving information using the information giving method to a plurality of images, a plurality of images with information added are accumulated,
Update the computing unit using a plurality of images to which the information is attached,
The information provision method according to claim 4 or 5. When calculating the probability using the computing unit constructed using an image to which the information of less than a predetermined number is assigned, and using an image to which the information of a predetermined number or more is provided. In the case of calculating the probability using the computing unit,
Changing the first threshold and the second threshold;
The information provision method according to claim 4 or 5. When calculating the probability using the computing unit constructed using an image to which the information of less than a predetermined number is given,
The first threshold value is set higher than the case where the probability is calculated using the computing unit constructed using an image to which a predetermined number or more of the information is added, and the second threshold value is set. Set it low,
The information providing method according to claim 7. In the acquisition, the device ID of the terminal is acquired from the terminal together with the image,
In the transmission, the device ID acquired together with the image is specified, and the image and the request reception information are transmitted to the terminal of the specified device ID among the one or more terminals.
The information providing method according to claim 1. In the acquisition of the image, the device ID of the terminal is acquired from the terminal together with the image,
In the transmission, the device ID acquired together with the image is specified, and the image and the request reception information are transmitted to a terminal different from the terminal of the specified device ID among the one or more terminals.
The information providing method according to claim 1. Furthermore, after processing whether to give information to the image by the information giving method, send information providing reward to the terminal,
The reward is when the probability for the image is greater than or equal to the second threshold and less than or equal to the first threshold, the probability is greater than the first threshold, and the probability is greater than the second threshold. Set higher than if it is below,
The information providing method according to claim 1. Before acquiring the image, accept a selection regarding the predetermined object to be photographed from the terminal;
The information providing method according to claim 1.   The information provision method according to claim 1, wherein at least one of the acquisition, the calculation, and the provision is performed by a processor. With a processor,
The processor is
Acquiring an image via a network from one or more terminals having a function of capturing an image or acquiring an image from another device;
For the acquired image, calculate the probability that a predetermined shooting object is included,
If the probability is above a first threshold, give the image information indicating the predetermined object to be photographed,
When the probability is lower than the second threshold, information indicating the predetermined photographing object is not given to the image,
If the probability is greater than or equal to the second threshold and less than or equal to the first threshold, the image and request acceptance information for accepting information assignment to the image may be any of the one or more terminals. Send to
Server device. A recording medium on which a program is recorded,
The program is stored in a computer.
Calculating a probability that a predetermined object to be captured is included in an image acquired via a network from one or more terminals having a function of capturing an image or a function of acquiring an image from another device;
If the probability is above a first threshold, give the image information indicating the predetermined object to be photographed,
When the probability is lower than the second threshold, information indicating the predetermined photographing object is not given to the image,
If the probability is greater than or equal to the second threshold and less than or equal to the first threshold, the image and request acceptance information for accepting information assignment to the image may be any of the one or more terminals. Send it to, execute the process,
recoding media.