JP5768647B2 - Image recognition system and image recognition method - Google Patents

Description

  The present invention relates to an image recognition system and an image recognition method.

  As one application example of image recognition, an image recognition system is known in which an input image is collated with a database and an object shown in the image is recognized. However, the amount of data stored in the database is enormous. Therefore, there is an increasing need for a network type (client / server type) image recognition system.

  In a network type image recognition system, a terminal device (client) transmits an image to be recognized to a server. Then, the server performs recognition processing and database collation for the image received from the terminal device. Then, the server returns the recognition result to the terminal device.

  In the above-described image recognition system, the terminal device can obtain a recognition result without having a large-capacity database. Moreover, if the database which a server has is updated appropriately, the terminal device can obtain the recognition result by the newest database.

  As a related technique, the following still image search transmission method has been proposed. In addition to still image data, the transmission-side storage means stores reduced image data for rough display of each still image and explanatory character information for the still image. The reduced image data and explanatory character information are transmitted to the receiving side. This is displayed on the receiving side, a required screen is selected, and the result is transmitted to the transmitting side as an image transmission request. (For example, Patent Document 1)

  As another related technique, the following image data communication method has been proposed. The captured data is accumulated, an INDEX image is created from the accumulated data, and the image is transmitted. The receiver selects only necessary data from the INDEX image, and notifies the sender side facsimile apparatus of the INDEX image and the necessary data. After the notification, the sender-side facsimile device searches the stored INDEX image and the notified data specified by the recipient-side facsimile device, and then receives only the data before processing of the notified data and related data. To the facsimile machine on the customer side. (For example, Patent Document 2)

JP-A-7-219967 JP 2004-104711 A

  In the network type image recognition system, as described above, the recognition target image is transmitted from the terminal device to the server. However, the image has a large amount of data. For this reason, network-type image recognition systems tend to have a slow recognition response.

  In particular, in a wireless network, the communication capacity of an uplink (a line for transmitting data from a terminal device to a server) is often significantly smaller than that of a downlink (a line for transmitting data from a server to a terminal device). Therefore, in this case, the recognition response may be further delayed.

  An object of the present invention is to improve the response of a network type image recognition system.

  An image recognition system according to one aspect of the present invention recognizes an image acquired by a terminal device in a server. The terminal device includes an image processing unit that generates a processed image from an input image, and a communication unit that transmits the processed image generated by the image processing unit to the server. The server includes a recognition unit that performs recognition processing on the processed image received from the terminal device, a shortage information determination unit that determines information that is insufficient in the recognition processing by the recognition unit, and the terminal device. And a request unit for requesting an image corresponding to the determination result by the lack information determination unit. The image processing unit generates the processed image from the input image based on a request from the server.

  According to the above aspect, the response of the network type image recognition system is improved.

It is a figure which shows the structure and function of the image recognition system of embodiment. It is a figure which shows the outline of an image recognition process. It is a figure which shows the Example of an image recognition process. It is a figure which shows an example of an input image. (A) is an example of an initial image, (b) (c) is a figure which shows an example of the image produced | generated according to the request | requirement from a server. It is a figure which shows the Example of request information. It is a figure explaining the function of an image processing part. It is a flowchart which shows operation | movement of a terminal device. It is a flowchart which shows operation | movement of a server. It is a flowchart which shows operation | movement of a server in the Example which recognizes a vehicle. It is a figure (the 1) which shows the procedure which specifies a vehicle name. It is FIG. (2) which shows the procedure which specifies a vehicle name. It is FIG. (The 3) which shows the procedure which pinpoints a vehicle name. It is FIG. (The 4) which shows the procedure which pinpoints a vehicle name. It is FIG. (5) which shows the procedure which specifies a vehicle name. It is FIG. (6) which shows the procedure which specifies a vehicle name. It is FIG. (7) which shows the procedure which specifies a vehicle name. It is a figure which shows the hardware constitutions of the computer system for implement | achieving a terminal device and a server.

  FIG. 1 is a diagram illustrating a configuration and functions of an image recognition system according to an embodiment. The image recognition system according to the embodiment includes a terminal device 1 and a server 2. The terminal device 1 and the server 2 are connected via a network 3. The network 3 may be a wireless network or a wired network. The network 3 may include a wireless network and a wired network.

  In FIG. 1, one terminal apparatus 1 is connected to the server 2, but the server 2 may accommodate a plurality of terminal apparatuses. That is, the server 2 may communicate with a plurality of terminal devices in parallel.

  The terminal device 1 includes an image acquisition unit 11, a storage device 12, an image processing unit 13, a communication unit 14, and a request processing unit 15. And the terminal device 1 acquires the image image | photographed with the camera 31. FIG.

  The camera 31 is a digital camera that acquires a still image or a digital video camera that acquires a moving image. The camera 31 may generate color image data or monochrome image data.

  The image acquisition unit 11 acquires an image generated by the camera 31. That is, the image acquisition unit 11 provides an interface between the camera 31 and the terminal device 1. Then, the image acquisition unit 11 stores the image acquired from the camera 31 in the storage device 12. Here, when the camera 31 outputs a still image, the image acquisition unit 11 stores the image acquired from the camera 31 in the storage device 12. When the camera 31 outputs a moving image, the image acquisition unit 11 may store an image at a designated timing in the moving image in the storage device 12. In the following description, an image acquired by the image acquisition unit 11 from the camera 31 may be referred to as an “input image”.

  The storage device 12 stores the image acquired by the image acquisition unit 11. The storage device 12 is realized by a semiconductor memory such as a flash memory, for example. The storage device 12 may be realized by other recording media such as a hard disk and an optical disk. Note that the recording medium of the storage device 12 may be detachable from the terminal device 1.

  The image processing unit 13 generates a processed image from the input image stored in the storage device 12. Here, the image processing unit 13 can generate a processed image from the input image based on a rule specified in advance. When the terminal device 1 receives the request information from the server 2, the image processing unit 13 generates a processed image from the input image based on the request information.

  The communication unit 14 transmits the processed image generated by the image processing unit 13 to the server 2. In addition, the communication unit 14 guides request information transmitted from the server 2 to the request processing unit 15. Further, the communication unit 14 receives the result of image recognition obtained by the server 2.

  The request processing unit 15 gives an instruction to the image processing unit 13 based on the request information transmitted from the server 2. Thereby, the image processing unit 13 can generate a processed image from the input image based on the request information transmitted from the server 2. That is, the image processing unit 13 can generate an image requested from the server 2.

  The server 2 includes a communication unit 21, a recognition unit 22, a database 23, a shortage information determination unit 24, and a request unit 25. Note that the server 2 may not include the database 23. That is, the database 23 may be provided outside the server 2. In this case, it is assumed that the server 2 accesses the database 23 with a predetermined interface.

  The communication unit 21 guides the image transmitted from the terminal device 1 (that is, the processed image generated by the image processing unit 13) to the recognition unit 22. In addition, the communication unit 21 transmits the request information generated by the request unit 25 to the terminal device 1. Furthermore, the communication unit 21 transmits the recognition result obtained by the recognition unit 22 to the terminal device 1.

  The recognition unit 22 performs a recognition process on the image received from the terminal device 1. At this time, the recognition unit 22 collates the received image with the image data registered in the database 23 as necessary. Various image data are registered in the database 23 according to the application. The image data registered in the database 23 is updated as necessary. And the recognition part 22 outputs a recognition result.

  The deficiency information determination unit 24 determines information that is insufficient in the recognition processing by the recognition unit 22. That is, the shortage information determination unit 24 specifies information necessary for the recognition unit 22 to make the recognition process successful.

  The request unit 25 generates request information representing an image corresponding to the determination result by the shortage information determination unit 24. This request information is transmitted to the terminal device 1 by the communication unit 21 as described above. That is, the request unit 25 requests the terminal device 1 for an image corresponding to the determination result by the shortage information determination unit 24.

  As described above, in the image recognition system of the embodiment, an image is transmitted from the terminal device 1 to the server 2, and the server 2 performs a recognition process. That is, the server 2 can operate as an image recognition device. If the terminal device 1 transmits the input image from the camera 31 to the server 2 as it is, not only information necessary for the recognition process but also information unrelated to the recognition process is transmitted from the terminal device 1 to the server 2. It is likely to be transmitted. In this case, the amount of data transmitted from the terminal device 1 to the server 2 increases, and the recognition response may be delayed. For this reason, in order to improve the recognition response, a method of suppressing transmission of information unrelated to the recognition process and transmitting only information necessary for the recognition process is preferable. However, it is difficult for the terminal device 1 to determine what information is necessary and what information is unnecessary in the recognition process by the server 2.

  Therefore, in the image recognition system of the embodiment, by adopting the following two processes, the communication load per time is reduced to improve the response until the recognition is completed. However, the image recognition system may perform only one of the following (1) and (2).

(1) The transmission server 2 only for necessary information through interactive exchange determines information (or an image) that is insufficient for recognizing or identifying a target object, and the terminal device 1 receives the information (or ), Image) extraction and transmission. The terminal device 1 generates or extracts insufficient information (features) based on a request from the server 2 and transmits only the generated or extracted information to the server 2. By this interactive exchange, transmission of information unnecessary for recognition processing is suppressed.

(2) Conversion to higher-order feature quantity The terminal device 1 converts the input image into a higher-order feature quantity and transmits it to the server 2. Thereby, the information amount of data transmission per time is reduced. As a result, speeding up of the response from the server 2 is realized, and the communication load on the uplink having a narrow bandwidth is reduced.

  FIG. 2 shows an outline of image recognition processing in the image recognition system of the embodiment. In the image recognition system of the embodiment, an image is transmitted from the terminal device 1 to the server 2, and the server 2 performs image recognition. In FIG. 2, it is assumed that the input image is already stored in the storage device 12 of the terminal device 1. The server 2 performs image recognition processing for recognizing an object shown in an image received from the terminal device 1 to a predetermined level.

  In the first (first) recognition process, the terminal device 1 generates a feature image having a smaller amount of information than the input image from the input image as an initial image. The feature image is, for example, an edge image of the input image. The edge image is generated, for example, by executing a Sobel filter operation on the input image. The feature image may be a reduced edge image of the input image. Then, the terminal device 1 transmits the generated feature image to the server 2. Note that the terminal device 1 may transmit a reduced image of the input image to the server 2 as the initial image.

  The server 2 performs a recognition process on the image received from the terminal device 1. However, the image received from the terminal device 1 in the first recognition process is a feature image in which the information amount is reduced compared to the input image in order to suppress the data amount. For this reason, when a recognition process is performed on this feature image, a desired recognition result may not be obtained. In the example illustrated in FIG. 2, it is assumed that the server 2 cannot recognize an object shown in the image up to a predetermined level in the recognition process for the feature image.

  In this case, the server 2 identifies information that is insufficient in the recognition process (that is, information necessary for recognizing an object shown in the image up to a predetermined level), and transmits an image necessary for the recognition process. Is requested to the terminal device 1. Thereafter, the image recognition system proceeds to the second recognition process.

  When the server 2 can recognize the object shown in the image from the above-described feature image to a predetermined level, the server 2 ends the recognition process. In this case, the server 2 transmits the recognition result to the terminal device 1.

  In the second recognition process, the terminal device 1 provides information that is lacking in the recognition process by the server 2 based on a request from the server 2. That is, the terminal device 1 generates an image requested from the server 2 and transmits it to the server 2.

  When the server 2 receives an image newly generated by the terminal device 1 in response to the above request, the server 2 performs recognition processing on the image. In this example, it is assumed that the server 2 cannot recognize the object shown in the image to a predetermined level even by the second recognition process. In this case, the server 2 specifies information that is insufficient in the recognition process, as in the first recognition process. However, the shortage information specified in the second recognition process is different from the shortage information specified in the first recognition process. Then, the server 2 requests the terminal device 1 to transmit an image specified in the second recognition process and necessary for the recognition process. Thereafter, the image recognition system proceeds to the third recognition process.

  The third and subsequent recognition processes are substantially the same as the second recognition process. However, the images requested by the server 2 to the terminal device 1 in each recognition process are different from each other. Therefore, images transmitted from the terminal device 1 to the server 2 in each recognition process are different from each other.

  The above recognition process is repeatedly executed by the terminal device 1 and the server 2 until an object shown in the image is recognized to a predetermined level. That is, the terminal device 1 and the server 2 request the necessary image from the server 2 to the terminal device 1 until the object shown in the image is recognized to a predetermined level. An interactive process including a procedure of generating an image corresponding to the request and transmitting it to the server 2 and a procedure of performing a recognition process on the image newly received by the server 2 are repeatedly performed.

  In the example shown in FIG. 2, the object shown in the image is recognized to a predetermined level in the Nth recognition process. Then, the server 2 transmits the recognition result to the terminal device 1. Accordingly, the terminal device 1 can receive the recognition result for the input image from the server 2.

  As described above, in the image recognition method of the embodiment, the terminal device 1 transmits a feature image having a smaller amount of information than the input image to the server 2 in the initial recognition process. In the second and subsequent recognition processes, the terminal device 1 transmits to the server 2 an image with the minimum amount of information determined to be insufficient in the previous recognition process. Therefore, compared with the method of transmitting the input image as it is to the server 2, the image recognition method of the embodiment suppresses the amount of data transmitted from the terminal device 1 to the server 2.

  Here, particularly in a wireless network, the capacity of an uplink (a line for transmitting data from the terminal apparatus 1 to the server 2) is significantly larger than that of a downlink (a line for transmitting data from the server 2 to the terminal apparatus 1). Often small. In such a network, when the amount of uplink data increases, the response of the image recognition system may deteriorate.

  However, according to the image recognition method of the embodiment, the amount of data transmitted from the terminal device 1 to the server 2 is suppressed as described above. Therefore, according to the image recognition method of the embodiment, the response of the image recognition system is improved particularly in a system in which the uplink capacity is small compared to the downlink.

  FIG. 3 shows an example of image recognition processing in the image recognition system of the embodiment. In this embodiment, the terminal device 1 is mounted on a vehicle such as an automobile. A camera 31 shown in FIG. 1 is also mounted on the same vehicle. The camera 31 acquires an image around the vehicle. Then, the terminal device 1 requests the server 2 to perform processing for recognizing an object (mainly another vehicle) shown in the image input from the camera 31.

  FIG. 4 shows an example of the input image. This input image is obtained by the camera 31 photographing a parking lot. In addition, one vehicle is shown in this input image. The input image is stored in the storage device 12 shown in FIG.

  The terminal device 1 generates a reduced edge image shown in FIG. 5A from the input image shown in FIG. The reduced edge image is generated by the image processing unit 13. At this time, the image processing unit 13 can generate an edge image of the reduced image after executing the process of reducing the input image. Alternatively, the image processing unit 13 may reduce the edge image after executing the process of generating the edge image of the input image. It is assumed that the reduction rate is designated in advance. The edge image is generated using, for example, an X-direction Sobel filter and a Y-direction Sobel filter. It is assumed that the size and coefficient of each Sobel filter are also specified in advance. The data amount of the reduced edge image generated in this way is significantly smaller than that of the input image. Then, the transmission unit 14 of the terminal device 1 transmits the reduced edge image generated as described above to the server 2.

  The server 2 receives the above-described reduced edge image from the terminal device 1. Then, the recognition unit 22 illustrated in FIG. 1 performs a recognition process on the received reduced edge image. In this embodiment, the recognition unit 22 performs a recognition process on the reduced edge image illustrated in FIG. 5A to determine that there is a high possibility that a vehicle is present in the image. The method for determining whether or not a vehicle is present in the edge image is not particularly limited, and a known algorithm may be used. For example, the recognition unit 22 may estimate the presence of the vehicle based on a combination of a plurality of edges extending in a substantially horizontal direction and a plurality of edges extending in a substantially vertical direction.

  However, in this embodiment, it is assumed that the recognition unit 22 cannot recognize the vehicle type and vehicle name from the reduced edge image. Then, the shortage information determination unit 24 shown in FIG. 1 specifies information necessary for recognizing the vehicle type and vehicle name of the vehicle shown in the image. In this embodiment, the recognizing unit 22 uses the image of the lower end of the vehicle in order to estimate the direction of the vehicle (front, side, diagonal, and rear) shown in the image. In this case, the shortage information determination unit 24 determines that “the image of the lower end portion of the vehicle is insufficient”.

  Then, the request unit 25 requests the terminal device 1 for an image of the lower end portion of the vehicle shown in the image. At this time, the request unit 25 uses, for example, request information to specify a rectangular area including the lower end of the vehicle. In order to reduce the amount of information, this rectangular area is, for example, the smallest rectangle that includes the lower end of the vehicle. In the example shown in FIG. 4, the request unit 25 specifies a rectangle R1. In this case, the request information includes coordinate information that specifies the position and size of the rectangular area. Then, the server 2 transmits this request information to the terminal device 1. That is, the server 2 requests the terminal device 1 for an image necessary for recognizing the vehicle type and vehicle name of the vehicle shown in the image.

  When receiving the above request information from the server 2, the terminal device 1 extracts the image specified by the request information from the input image shown in FIG. That is, the image processing unit 13 generates a partial image shown in FIG. 5B including the lower end portion of the vehicle by extracting an image inside the rectangle R1 from the input image. Then, the terminal device 1 transmits a partial image including the lower end portion of the vehicle to the server 2.

  The server 2 receives a partial image including the lower end portion of the vehicle from the terminal device 1. Then, the recognition unit 22 performs a recognition process on the received partial image. In this embodiment, the recognition unit 22 estimates the direction of the vehicle shown in the image by performing recognition processing on the partial image including the lower end of the vehicle.

  When using an image including the lower end of the vehicle, the vehicle orientation can be estimated by detecting parts (emblem, front grill, license plate, light, tire, door, etc.) corresponding to the vehicle orientation. . For example, the direction of the vehicle is estimated by detecting a license plate and / or a tire. Since the license plate exists only in the front and rear of the vehicle, the rough direction (forward, backward, diagonal) of the vehicle is estimated by detecting a substantially rectangular shape with a long side in the lateral direction (that is, the license plate). it can. In addition, when the vehicle is viewed from the side, the tires appear to be approximately circular or approximately oval. Therefore, by detecting such a shape (that is, the tire), it is possible to estimate the rough direction of the vehicle (sideways, oblique). . That is, when the license plate is detected without detecting the tire, the vehicle direction is estimated to be forward or backward. Further, when the tire is detected without detecting the license plate, it is estimated that the vehicle is oriented sideways. Further, when the license plate and the tire are detected, it is estimated that the vehicle is inclined.

  In this way, the server 2 estimates the orientation of the vehicle shown in the image based on the partial image including the lower end portion of the vehicle. However, in this embodiment, it is assumed that the recognition unit 22 cannot recognize the vehicle type and vehicle name of the vehicle. Then, the shortage information determination unit 24 specifies other information necessary for recognizing the vehicle type and vehicle name of the vehicle shown in the image. In this embodiment, it is assumed that the recognition unit 22 uses the image of the center of the vehicle in order to recognize the vehicle type and vehicle name of the vehicle shown in the image. In this case, the shortage information determination unit 24 determines that “the image at the center of the vehicle is insufficient”.

  If it does so, the request | requirement part 25 will request | require the image of the center part of the vehicle currently reflected in the image with respect to the terminal device 1. FIG. At this time, the request unit 25 uses, for example, request information to designate a rectangular area including the central part of the vehicle. In order to reduce the amount of information, this rectangular area is, for example, the smallest rectangle that includes the center of the vehicle. In the example illustrated in FIG. 4, the request unit 25 designates a rectangle R2. Also in this case, the request information includes coordinate information specifying the position and size of the rectangular area. Then, the server 2 transmits this request information to the terminal device 1. That is, the server 2 requests the terminal device 1 for another image necessary for recognizing the type and name of the vehicle shown in the image.

  When receiving the above request information from the server 2, the terminal device 1 extracts the image specified by the request information from the input image shown in FIG. That is, the image processing unit 13 extracts the image inside the rectangle R2 from the input image, thereby generating a partial image shown in FIG. 5C including the central portion of the vehicle. Then, the terminal device 1 transmits a partial image including the lower end portion of the vehicle to the server 2.

  The server 2 receives a partial image including the central portion of the vehicle from the terminal device 1. Then, the recognition unit 22 performs a recognition process on the received partial image. In this embodiment, the recognition unit 22 can identify the vehicle type, manufacturer, and vehicle name of the vehicle shown in the image by performing recognition processing on the partial image including the central portion of the vehicle. To do.

  Here, consider the case where the vehicle type and direction of the vehicle shown in the image are specified by the previous image processing. For example, it is assumed that the vehicle type is “sedan”. In this case, an emblem is often attached to the front surface of the vehicle. Therefore, when the direction of the vehicle is “forward”, it is highly possible that the manufacturer can be specified based on the emblem. In many cases, a vehicle name (model name) is displayed at the rear of the vehicle. Therefore, when the direction of the vehicle is “backward”, it is highly possible that the vehicle name (model name) can be specified by performing character recognition on the image.

  In this manner, the server 2 specifies the vehicle type, manufacturer, and vehicle name of the vehicle shown in the image based on the partial image including the central portion of the vehicle. Then, the server 2 transmits the recognition result to the terminal device 1.

  FIG. 6 shows an example of request information used by the server 2 to request an image from the terminal device 1. The request information includes “recognition completion flag”, “image type”, “processing range”, and “reduction rate”. The request information is created by the request unit 25 based on the determination result of the shortage information determination unit 24.

  The recognition completion flag indicates whether or not the recognition process by the recognition unit 22 has been completed. When the recognition unit 22 obtains a predetermined recognition result, “recognition completion flag = 1” is generated, and when the predetermined recognition result is not obtained, “recognition completion flag = 0” is generated. In the example shown in FIGS. 3 to 5, it is determined that the recognition process has been completed when the name of the vehicle shown in the image is specified.

  The image type information represents the type of image that the server 2 needs in the recognition process. In this embodiment, when the server 2 requests an original image (color image), “image type = 0” is set. When a gray image is requested, “image type = 1” is set, and an edge image is requested. To do so, “image type = 2” is set.

  The processing range information represents an image range (that is, an image region) that the server 2 needs in recognition processing. In this embodiment, the processing range information can specify the size and position of a rectangular area. In this case, the processing range information specifies, for example, the coordinates of the upper left vertex and the lower right vertex of the requested rectangular image area. Alternatively, the processing range information may specify the coordinates of the four vertices of the requested rectangular image area. Note that the shape of the image area that the server 2 can request from the terminal device 1 is not limited to a rectangle.

  The reduction rate information represents the size of an image that the server 2 needs in the recognition process. The reduction ratio that can be specified is, for example, 1/2, 1/4, or the like of the original image. However, the reduction rate information may be set so that an arbitrary reduction rate can be specified in the range of 0 to 100 percent.

The request information may include other information. For example, the request information may include information specifying a filter coefficient for generating an edge image and information specifying an edge type (for example, an edge image from which an edge in a diagonal direction is extracted).
The request information is transmitted from the server 2 to the terminal information 1. Then, the image processing unit 13 of the terminal device 1 generates a processed image according to the request information.

  FIG. 7 is a diagram illustrating the function of the image processing unit 13. The image processing unit 13 includes an image memory 41, a gray image generation unit 42, an edge image generation unit 43, a region extraction unit 44, and an image reduction unit 45. Then, the image processing unit 13 acquires an input image from the storage device 12. This input image (that is, the original image) is temporarily stored in the image memory 41.

  When a gray image is requested from the server 2, the gray image generation unit 42 generates a gray image from the color image stored in the image memory 41. The gray image generated by the gray image generation unit 42 is stored in the image memory 41.

  When an edge image is requested from the server 2, the edge image generation unit 43 generates an edge image from the image stored in the image memory 41. Here, the edge image generation unit 43 includes an X-direction Sobel filter 43x and a Y-direction Sobel filter 43y. That is, the edge image generation unit 43 can generate a horizontal edge image and a vertical edge image. The edge image generated by the edge image generation unit 43 is stored in the image memory 41.

  When the processing range information is set in the request information received from the server 2, the region extraction unit 44 extracts an image within the designated processing range from the image stored in the image memory 41. The image extracted by the region extraction unit 44 (that is, the partial image) is stored in the image memory 41.

  When the reduction rate information is set in the request information received from the server 2, the image reduction unit 45 reduces the image stored in the image memory 41 according to the specified reduction rate. The reduced image generated by the image reduction unit 45 is stored in the image memory 41.

  The image processing unit 13 outputs the processed image generated by the gray image generation unit 42, the edge image generation unit 43, the region extraction unit 44, or the image reduction unit 45. However, the image processing unit 13 can output a processed image generated using two or more of the gray image generation unit 42, the edge image generation unit 43, the region extraction unit 44, and the image reduction unit 45. For example, when a reduced edge image is requested from the server 2, the image processing unit 13 uses the edge image generation unit 43 and the image reduction unit 45 to generate the requested image. At this time, for example, the edge image generation unit 43 generates an edge image from the input image, and then the image reduction unit 45 generates a reduced edge image from the edge image. Alternatively, the image reduction unit 45 may generate a reduced image from the input image, and then the edge image generation unit 43 may generate a reduced edge image from the reduced image.

  Thus, the image processing unit 13 generates a processed image from the input image in accordance with the request information received from the server 2. That is, the image processing unit 13 generates an image requested from the server 2. Then, the terminal device 1 transmits the processed image generated by the image processing unit 13 (that is, the image requested from the server 2) to the server 2.

  FIG. 8 is a flowchart showing the operation of the terminal device 1 in the image recognition system of the embodiment. The process of this flowchart is executed when, for example, an instruction to start image recognition is given to the terminal device 1. An instruction to start image recognition is given by a user, for example.

  In step S <b> 1, the image processing unit 13 acquires an input image from the storage device 12. It is assumed that the storage device 12 stores an image generated by the camera 31 at this time.

  In step S <b> 2, the request processing unit 15 determines whether the terminal device 1 has received request information from the server 2. If the terminal device 1 has not received the request information from the server 2, the processing of the terminal device 1 proceeds to step S3. On the other hand, if the terminal device 1 has received the request information from the server 2, the processing of the terminal device 1 proceeds to step S4. Note that in the initial operation state immediately after the processing of this flowchart is started, the terminal device 1 has not received the request information from the server 2, and therefore the processing of step S3 is executed.

  In step S3, the image processing unit 13 generates an initial image from the input image according to a rule specified in advance. This rule is specified from the server 2, for example. Alternatively, this rule may be determined by negotiation between the terminal device 1 and the server 2. As an example, “image type = 2 (edge image)” and “reduction ratio = 1/4” are designated. In this case, the image processing unit 13 generates an edge image reduced to ¼ from the input image.

  If the terminal device 1 has received the request information from the server 2, the process of step S4 is executed. In step S <b> 4, the request processing unit 15 gives a rule for a processed image generation method to the image processing unit 13 in accordance with the received request information. Then, the image processing unit 13 generates a processed image from the input image according to the rules given from the request processing unit 15. For example, when the “processing range” is set in the request information from the server 2, the image processing unit 13 extracts an image inside the processing range set in the request information from the input image, The image requested from the server 2 is generated.

  In step S5, the communication unit 14 transmits the image generated by the image processing unit 13 in step S3 or S4 to the server 2. That is, in the first recognition process, the communication unit 14 transmits the initial image generated in step S3 to the server 2. In the second and subsequent recognition processes, the communication unit 14 transmits the image generated in step S <b> 4 to the server 2 in response to a request from the server 2.

  The server 2 executes a recognition process on the image received from the terminal device 1. Then, the server 2 creates request information according to the status or result of the recognition process, and transmits the request information to the terminal device 1.

  In steps S <b> 6 to S <b> 7, the terminal device 1 receives request information from the server 2. The request information is interpreted by the request processing unit 15. If the recognition completion flag is “1: completion”, the processing of the terminal device 1 is terminated. Although omitted in FIG. 8, when the recognition completion flag is “1: completed”, the terminal device 1 also receives a recognition result from the server 2. On the other hand, if the recognition completion flag is “0: incomplete”, the processing of the terminal device 1 proceeds to step S4.

  Thus, the terminal device 1 transmits an initial image in steps S3 and S5. Thereafter, the terminal device 1 repeatedly executes the processes in steps S4 to S7 until the recognition process in the server 2 is completed. That is, every time an image is requested from the server 2, the terminal device 1 generates the requested image and transmits it to the server 2.

  FIG. 9 is a flowchart showing the operation of the server 2 in the image recognition system of the embodiment. The process of this flowchart is executed when the server 2 receives a request for image recognition from the terminal device 1, for example.

In step S <b> 11, the communication unit 21 receives an image from the terminal device 1. At this time, the server 2 receives the image generated in step S3 or S4 in FIG.
In steps S <b> 12 to S <b> 13, the recognition unit 22 performs a recognition process on the image received from the terminal device 1. At this time, the recognition unit 22 may collate with the database 23 as necessary. If the recognition process has not been completed, the process of the server 2 proceeds to step S14. On the other hand, when the recognition process is completed, the process of the server 2 proceeds to step S16. For example, in the embodiment shown in FIG. 3, the image recognition process is intended to specify the vehicle type and vehicle name of the vehicle shown in the image. In this case, it is determined that the recognition process has been completed when the vehicle type and vehicle name of the vehicle shown in the image are specified.

  In step S <b> 14, the shortage information determination unit 24 determines information that is insufficient in the recognition process performed by the recognition unit 22. That is, the shortage information determination unit 24 determines what kind of image is necessary to complete the recognition process. For example, in the embodiment shown in FIG. 3, it is determined that “the image at the lower end of the vehicle is insufficient” or “the image at the center of the vehicle is insufficient”.

  In step S <b> 15, the request unit 25 requests the terminal device 1 for missing information based on the determination result by the shortage information determination unit 24. That is, the server 2 requests the terminal device 1 for an image necessary for completing the recognition process. This request is realized by transmitting the request information shown in FIG.

  When receiving the request information from the server 2, the terminal device 1 generates the requested image and transmits it to the server 2 as described with reference to FIG. 8. Then, the server 2 receives the image. That is, the process of the server 2 returns to step S11 after executing step S15.

  When the recognition process by the recognition unit 22 is completed (step S13: YES), the process of step S16 is executed. In step S <b> 16, the server 2 transmits the recognition result by the recognition unit 22 to the terminal device 1. At this time, in addition to the recognition result, the server 2 transmits request information in which “completion flag = 1 (completion)” is set to the terminal device 1.

  Thus, whenever the server 2 receives an image from the terminal device 1, the server 2 performs recognition processing on the image. When the recognition process has not been completed, the server 2 requests the terminal device 1 for an image necessary for completing the recognition process. That is, the server 2 requests the terminal device 1 for a necessary image by repeatedly executing steps S11 to S15 until the recognition process is completed.

  At this time, the server 2 requests a minimum image necessary for the recognition process, for example. Then, the terminal device 1 substantially transmits only the requested image to the server 2. Therefore, the amount of data transmitted from the terminal device 1 to the server 2 is greatly reduced as compared with the method of transmitting the input image as it is. As a result, the response of the image recognition system is improved.

<Example>
In the following embodiments, the image recognition system is used in an application in which the name of a vehicle photographed by a camera is specified and replaced with a CG image. Specifically, parking assistance that can grasp the 360-degree surrounding situation by attaching wide-angle cameras to the front, rear, left and right of the host vehicle, and combining and displaying the images of these four cameras in one image Assume a system. Here, in order to provide a 360-degree all-around situation, for example, the synthesized image is converted into an image (overhead video) viewed from directly above the host vehicle. At this time, since the bird's-eye view video is created by converting and synthesizing the images of the four cameras on a two-dimensional plane, an object around the host vehicle is stretched. For this reason, if such an image is displayed as it is, it becomes difficult to visually recognize the presence of the other vehicle, and it becomes difficult to grasp the positional relationship of the other vehicle with respect to the own vehicle. Therefore, in the parking support system of this embodiment, the image of the vehicle stretched by the image conversion is replaced with a CG image (or displayed in a superimposed manner).

  In order to replace the image of the vehicle shown in the original image with the CG image, it is necessary to specify the vehicle name of the vehicle. Here, when specifying the vehicle name of the vehicle shown in the image, two methods are conceivable. In the first method, a terminal device mounted on the host vehicle performs all image recognition processing. In the second method, a terminal device mounted on the host vehicle requests an image recognition process from the server.

  According to the first method, since the communication process does not occur, the recognition response is high speed. However, in the first method, it is necessary to store a huge amount of data necessary for specifying the vehicle type and the vehicle name on the terminal side. Moreover, it is necessary for the terminal side to update data in response to a vehicle model change or the like, which is cumbersome for the user.

  In the second method, it is easy to store a huge amount of data for image collation and to deal with a vehicle model change. However, in the second method, wireless communication is performed between the terminal device mounted on the vehicle and the server. Here, the image has a large data capacity, and generally, the wireless communication has a small uplink communication capacity. For this reason, the time required for wireless communication between the terminal device and the server becomes long, and the recognition response may be reduced. Therefore, by applying the method of the embodiment of the present invention to the parking assistance system described above, the problem of the second method is solved or alleviated, and the recognition response is improved.

  FIG. 10 is a flowchart showing the operation of the server 2 in the embodiment for recognizing a vehicle. The process of this flowchart is executed when the server 2 receives a request for image recognition from the terminal device 1, for example.

  In step S <b> 21, the communication unit 21 receives an image from the terminal device 1. At this time, the server 2 receives the processed image generated in the terminal device 1. The initial image that the server 2 first receives from the terminal device 1 is, for example, a reduced edge image of the input image from the camera 31.

  In steps S <b> 22 to S <b> 23, the recognition unit 22 determines whether a vehicle is present in the image received from the terminal device 1. At this time, for example, as described with reference to FIG. 4, the recognition unit 22 estimates the presence of the vehicle based on a combination of a plurality of edges extending in a substantially horizontal direction and a plurality of edges extending in a substantially vertical direction. be able to.

  When it is determined that there is a high possibility that a vehicle is present in the image, the process of the server 2 proceeds to step S24. On the other hand, when it is determined that the possibility that a vehicle is present in the image is low, not only a case where the vehicle does not actually exist but also a case where the vehicle cannot be detected because the amount of information in the image is insufficient. Therefore, the server 2 requests the shortage information from the terminal device 1 in step S28. In this case, the server 2 may request an image in which the reduction rate and the processing range are changed according to the degree of detectability.

  In steps S <b> 24 to S <b> 25, the recognition unit 22 estimates the direction of the vehicle shown in the image received from the terminal device 1. And when the direction of a vehicle is specified, the process of the server 2 progresses to step S26. On the other hand, when the direction of the vehicle cannot be specified, the server 2 requests the shortage information from the terminal device 1 in step S28. Here, for example, as described with reference to FIG. 4, the recognition unit 22 detects parts (emblem, front grill, license plate, light, tire, door, etc.) corresponding to the direction of the vehicle, The direction of the vehicle can be estimated. Therefore, when the server 2 cannot identify the orientation of the vehicle from the previously received image, the terminal device 1 detects the vehicle emblem, front grill, license plate, light, tire, door, and the like. For the image of the lower end of the vehicle.

  In steps S <b> 26 to S <b> 27, the recognition unit 22 uses the database 23 to perform processing for specifying the vehicle name of the vehicle shown in the image received from the terminal device 1. When the vehicle name can be specified, the server 2 transmits a CG image corresponding to the specified vehicle name to the terminal device 1 as a recognition result in step S29. At this time, the server 2 also transmits a message indicating that the recognition process has been completed to the terminal device 1. This message is realized by the recognition completion flag shown in FIG.

  On the other hand, when the vehicle name of the vehicle cannot be specified, the server 2 requests the shortage information from the terminal device 1 in step S28. In this case, for example, the server 2 may request the terminal device 1 for an image with a small reduction rate (an image with a large size).

  In addition, when the image which reflected the vehicle from the front is acquired, the recognition part 22 can identify a vehicle type and a vehicle name based on an emblem, a front grille, a license plate, a light, etc. Moreover, when the image which reflected the vehicle from the back is acquired, the recognition part 22 can specify a vehicle type and a vehicle name based on an emblem, a license plate, a light, a vehicle model number display, etc. Furthermore, when the image which reflected the vehicle from the side is acquired, the recognition part 22 can specify a vehicle type and a vehicle name based on the number of doors (2 doors / 4 doors), the shape of a door, etc. . Thus, there is a correspondence between the direction of the vehicle and the parts used to specify the vehicle type and vehicle name of the vehicle. That is, if the direction of the vehicle is specified before the parts for specifying the vehicle type and the vehicle name are detected, the parts for specifying the vehicle type and the vehicle name can be easily detected. Therefore, in this embodiment, the direction of the vehicle is specified before specifying the vehicle type and the vehicle name. And the response of the process which specifies a vehicle type and a vehicle name improves by employ | adopting this procedure.

  The server 2 repeatedly executes the processes of steps S21 to S28 until the vehicle name of the vehicle shown in the image can be specified. However, after the presence of the vehicle is confirmed, the processes in steps S23 to S24 are skipped. Moreover, after the direction of the vehicle is specified, the processes in steps S23 to S26 are skipped.

  When the vehicle name cannot be specified within a predetermined time from the start of the image recognition process, the server 2 ends the recognition process and transmits the recognition result obtained at that time to the terminal device 1. For example, when the model of the vehicle shown in the image (for example, a sedan) is specified, the server 2 extracts a CG image of a typical sedan-type vehicle from the database 23 and transmits it to the terminal device 1. . Further, when a color image is received from the terminal device 1, the server 2 may attach the same color as the vehicle shown in the image to the CG image and transmit it to the terminal device 1.

  FIGS. 11-17 is a figure which shows in detail an example of the process which specifies the vehicle name of the vehicle currently reflected in the image. This process corresponds to steps S21 to S28 shown in FIG. In addition, when the information describing the procedure of the process for specifying the vehicle name is stored in the database, and the server 2 specifies the vehicle name according to the procedure, FIGS. 11 to 17 show the structure of the database. .

  In the example illustrated in FIGS. 11 to 17, the server 2 receives the reduced edge image as the initial image from the terminal device 1, and determines that there is a high possibility that a vehicle exists in the image based on the initial image. It shall be. Therefore, in the following, the description will be started from step S31 shown in FIG.

  In step S31, the server 2 requests the terminal device 1 for a reduced edge image including the lower end portion of the vehicle in order to detect the license plate and the tire. In step S31, the server 2 requests a reduced edge image having a higher resolution than the initial image received earlier.

  In step S32, the server 2 estimates the direction of the vehicle using the reduced edge image corresponding to the request in step S31. Here, when the license plate is detected without detecting the tire, the server 2 determines that the vehicle is “forward or backward”. When the tire is detected without detecting the license plate, the server 2 determines that the vehicle is “sideways”. When the license plate and the tire are detected, the server 2 determines that the vehicle is “oblique”.

  When the direction of the vehicle cannot be specified in step S32, the server 2 requests the terminal device 1 for a new image in step S33. At this time, the server 2 requests the terminal device 1 for an image that is the same area as the image requested in step S31 and has a low reduction rate (that is, a large size). Then, the server 2 estimates the direction of the vehicle using the edge image corresponding to the request in step S33.

  If the orientation of the vehicle cannot be specified in step S34, the server 2 requests the terminal device 1 for a reduced edge image of the images taken at different times in step S35. In other words, the server 2 performs recognition processing on images taken at different times.

  When it is determined in step S32 or S34 that the vehicle is “forward or backward”, the process of the server 2 proceeds to step S41 shown in FIG. The description of the operation when the vehicle is determined to be “laterally” or “oblique” will be omitted.

  In step S <b> 41, the server 2 executes a process for identifying the vehicle type and direction of the vehicle shown in the image with respect to the previously received initial image. Here, as a method of identifying the vehicle type from the appearance of the vehicle, for example, a method of determining based on the degree of coincidence between the appearance image of the vehicle and the model image is conceivable. In this case, for example, images of sedans, wagons, vans, trucks, buses, light vehicles, and the like are registered in the database 23 as model images. The recognizing unit 22 calculates the degree of coincidence between the appearance image of the vehicle and each model image. The vehicle type corresponding to the model image having the highest degree of coincidence with the appearance image of the vehicle and the degree of coincidence being higher than a predetermined threshold (for example, 75%) is the vehicle type of the vehicle shown in the image. Determined.

  The server 2 may determine whether the vehicle is “forward” or “rearward” in the process of identifying the vehicle type. In this case, as a model image, a sedan, a wagon, a van, a truck, a bus, and a light vehicle may each be registered in the database 23 when viewed from the front and when viewed from the rear.

  If the vehicle type and vehicle orientation cannot be identified in step S41, the server 2 requests the terminal device 1 for an edge image having a lower reduction rate than the initial image in step S42. In step S43, the server 2 estimates the vehicle type and the vehicle direction using the edge image corresponding to the request in step S42.

  When it is determined that the vehicle shown in the image is a sedan and is forward-facing, the processing of the server 2 proceeds to step 51 shown in FIG. When it is determined that the vehicle shown in the image is a sedan and is facing backward, the processing of the server 2 proceeds to step 71 shown in FIG. In addition, description is abbreviate | omitted about the process when the vehicle reflected in the image is not a sedan. If the vehicle type and the vehicle direction cannot be identified in step S41 or S43, the process of the server 2 proceeds to step S81 shown in FIG.

  In step S51, the server 2 requests the terminal device 1 for an edge image of the central portion of the vehicle in order to detect the emblem. In step S52, the server 2 detects an emblem from the image corresponding to the request in step S51, and identifies the manufacturer of the vehicle from the emblem. Here, it is assumed that a template image of an emblem of each manufacturer is registered in the database 23. And the recognition part 22 specifies the manufacturer of the vehicle reflected in the image by comparing the emblem image extracted from the received image with each template image.

  If the manufacturer cannot be specified in step S52, the server 2 requests a new image from the terminal device 1 in step S53. In this case, the server 2 requests an image with a low reduction rate within the same range as the image requested in step S51. In step S54, the server 2 specifies the manufacturer of the vehicle using the image corresponding to the request in step S53.

  If the manufacturer of the vehicle cannot be specified in step S54, the server 2 requests the terminal device 1 for a reduced edge image of the images taken at different times in step S55. In other words, the server 2 performs recognition processing on images taken at different times.

  After specifying the vehicle type and manufacturer in step S52 or S54, the processing of the server 2 proceeds to step S61 in FIG. In step S61, the server 2 requests the terminal device 1 for a reduced edge image from the center to the lower end of the vehicle in order to detect the front grill. In step S62, the recognizing unit 22 identifies the vehicle name by recognizing a character string (such as alphabets and numbers) existing on the front grille from the image corresponding to the request in step S61.

  If the vehicle name cannot be specified in step S62, the server 2 requests a new image from the terminal device 1 in step S63. In this case, the server 2 requests an image with a low reduction rate within the same range as the image requested in step S61. In step S64, the server 2 specifies the vehicle name using the image corresponding to the request in step S63.

  If the vehicle name cannot be specified in step S64, the server 2 requests the terminal device 1 for an edge image of an image taken at a different time in step S65. In other words, the server 2 performs recognition processing on images taken at different times.

  When it is determined that the vehicle shown in the image is a sedan and is facing backward, the processing of the server 2 proceeds to step 71 in FIG. In step S71, in order to detect the display of the model name of the vehicle, the server 2 requests the terminal device 1 for a reduced edge image of a portion where the display of the model name is expected to exist. Note that the position where the model name display is expected to be present is estimated based on the vehicle type specified in step S41 or S43, for example. In step S72, the recognizing unit 22 identifies the vehicle name by detecting the display of the vehicle model name based on the image corresponding to the request in step S71.

  If the vehicle name cannot be specified in step S72, the server 2 requests a new image from the terminal device 1 in step S73. In this case, the server 2 requests an image with a low reduction rate within the same range as the image requested in step S71. And in step S74, the server 2 specifies a vehicle name using the image corresponding to the request | requirement of step S73.

  If the vehicle name could not be specified in step S74, the server 2 requests the edge image of the central portion of the vehicle from the terminal device 1 to detect the emblem in step S75. Since the subsequent processing is substantially the same as the procedure described with reference to FIGS. 13 and 14, the description thereof will be omitted.

  When the vehicle type cannot be specified in steps S41 and S43, the process of the server 2 proceeds to step S81 shown in FIG. In step S81, in order to detect the license plate, the server 2 requests the terminal device 1 for a reduced edge image and color information of an area where the license plate is expected to exist. In step S82, the recognition unit 22 detects the license plate from the image corresponding to the request in step S81, and further detects the vehicle type using the color information. Here, the following three methods can be considered as a method of identifying the vehicle type based on the license plate. Note that the recognition unit 22 recognizes the vehicle type by any one of the following three methods or a combination of two or more.

(A) License plate classification number In the license plate display, the classification number on the right side of the place name indicates the purpose of the vehicle (cargo, passenger, special, large special) and the size of the vehicle (normal, small). means. For example, 1 series represents an ordinary freight car, 3 series represents an ordinary passenger car, 5 series represents a small passenger car, and 8 series represents a special purpose automobile. Therefore, the recognition part 22 can identify a vehicle model by recognizing the number on a license plate.

(B) Color of license plate The color and color of the background and letters of the license plate are determined according to the vehicle type. For example, the license plate of a light vehicle is determined to be black on a yellow background or yellow on a black background. In addition, the number plate of a car is determined to be green on a white background, or green and white. Therefore, the recognition unit 22 can identify the automobile and the light vehicle based on the color information (yellow, black, green, white).

(C) The size of the license plate The size of the license plate is classified into a large scale plate, a medium scale plate, and a small scale plate. For example, a vehicle having a large signboard is a large truck, a large bus, a special vehicle, or the like. Therefore, the vehicle type can be estimated by detecting the license plate and calculating its size.

  If the vehicle type cannot be specified in step S82, the server 2 requests a new image from the terminal device 1 in step S83. In this case, the server 2 requests an image with a low reduction ratio in the image and range requested in step S81. In step S84, the server 2 specifies the vehicle type using the image corresponding to the request in step S83.

  If the vehicle type could not be specified in step S84, the server 2 requests the terminal device 1 for a reduced edge image taken at a different time in step S85. In other words, the server 2 performs recognition processing on images taken at different times.

  After specifying the vehicle type in step S82 or S84, the server 2 requests the terminal device 1 for an edge image of the central portion of the vehicle in order to detect the emblem in step S91. Subsequent processing is substantially the same as the procedure described with reference to FIGS. Further, the server 2 may execute the procedure described with reference to FIG.

  Thus, the server 2 specifies the vehicle name of the vehicle shown in the image received from the terminal device 1 in the procedure shown in FIGS. At this time, the server 2 may realize identification of the vehicle name by executing a program describing the procedure shown in FIGS. Alternatively, information for realizing the procedure shown in FIG. 11 to FIG. 17 (that is, information for instructing the state of recognition processing and the processing content according to the result) may be registered in the database 23. In this configuration, the server 2 can know the processing content according to the recognition state of the parts of the vehicle and the information requested to the terminal side by referring to the database 23, so that the terminal device 1 and the server 2 Efficient interactive exchange is possible. As a result, since the server 2 can specify the vehicle name with a small amount of information, the transmission amount of information can be suppressed and the recognition response can be improved.

<Hardware configuration of image recognition device>
FIG. 18 is a diagram illustrating a hardware configuration of a computer system for realizing the terminal device 1 and the server 2. The hardware configuration of the computer system for realizing the terminal device 1 and the computer system for realizing the server 2 are almost the same. Therefore, hereinafter, the computer system for realizing the terminal device 1 and the computer system for realizing the server 2 will be referred to as a computer system 100. However, the capacity of the CPU, the capacity of the memory / storage device, and the like are different from each other in the terminal device 1 and the server 2.

  As shown in FIG. 18, the computer system 100 includes a CPU 101, a memory 102, a storage device 103, a reading device 104, a communication interface 106, and an input / output device 107. The CPU 101, the memory 102, the storage device 103, the reading device 104, the communication interface 106, and the input / output device 107 are connected to each other via a bus 108, for example.

  In the terminal device 1, the CPU 101 can provide some or all of the functions of the image processing unit 13 and the request processing unit 15 by executing an image recognition program using the memory 102. At this time, the CPU 101 can execute a program describing the processing of the flowchart shown in FIG.

  In the server 2, the CPU 101 can provide a part or all of the functions of the recognition unit 22, the shortage information determination unit 24, and the request unit 25 by executing an image recognition program using the memory 102. it can. At this time, the CPU 101 can execute a program describing the processing of the flowcharts shown in FIGS. 9, 10, and 11 to 17.

  The memory 102 is a semiconductor memory, for example, and includes a RAM area and a ROM area. The storage device 103 is, for example, a hard disk, and stores an image recognition program related to image recognition according to the embodiment. Note that the storage device 103 may be a semiconductor memory such as a flash memory. The storage device 103 may be an external recording device.

  In the terminal device 1, the storage device 12 illustrated in FIG. 1 is realized by the memory 102 or the storage device 103. In the server 2, the database 23 may be realized using the storage device 103.

  The reading device 104 accesses the removable recording medium 105 in accordance with an instruction from the CPU 101. The detachable recording medium 105 includes, for example, a semiconductor device (USB memory or the like), a medium to / from which information is input / output by a magnetic action (magnetic disk or the like), a medium to / from which information is input / output by an optical action (CD-ROM, For example, a DVD). The communication interface 106 transmits / receives data via a network according to instructions from the CPU 101. The input / output device 107 corresponds to, for example, a device that receives an instruction from a user, an interface that receives image data from a digital camera, an interface that outputs a recognition result, and the like.

The image recognition program of the embodiment is provided to the computer system 100 in the following form, for example.
(1) Installed in advance in the storage device 103.
(2) Provided by the removable recording medium 105.
(3) Provided from the program server 110.

The following additional notes are further disclosed with respect to the embodiments including the examples described above.
(Appendix 1)
An image recognition system for recognizing an image acquired by a terminal device at a server,
The terminal device
An image processing unit for generating a processed image from the input image;
A communication unit that transmits the processed image generated by the image processing unit to the server,
The server
A recognition unit that performs recognition processing on the processed image received from the terminal device;
A deficiency information determination unit for determining deficient information in the recognition processing by the recognition unit;
A request unit for requesting an image corresponding to a determination result by the shortage information determination unit to the terminal device,
The image processing unit generates the processed image from the input image based on a request from the server.
(Appendix 2)
The image processing unit generates a first processed image having a smaller amount of information than the input image from the input image before the terminal device receives a request from the server,
The recognition unit performs recognition processing on the first processed image,
The insufficient information determination unit determines information that is insufficient in the recognition processing for the first processed image by the recognition unit,
The image recognition system according to appendix 1, wherein the request unit requests the terminal device for an image corresponding to information that is insufficient in the recognition process for the first processed image.
(Appendix 3)
The image recognition system according to appendix 2, wherein the first processed image is an edge image of the input image.
(Appendix 4)
The image recognition system according to appendix 2, wherein the first processed image is a reduced image of the input image.
(Appendix 5)
The image recognition system according to claim 1, wherein the request unit requests the terminal device for an image in a range corresponding to a determination result by the lack information determination unit.
(Appendix 6)
The image recognition system according to appendix 1, wherein the request unit requests the terminal device for an image of an image type corresponding to a determination result by the shortage information determination unit.
(Appendix 7)
The image recognition system according to appendix 1, wherein the request unit requests the terminal device for an image with a reduction ratio corresponding to a determination result by the shortage information determination unit.
(Appendix 8)
The image processing unit generates a second processed image from the input image based on a request from the server,
The recognition unit performs a recognition process on the second processed image,
The insufficient information determination unit determines information that is insufficient in the recognition processing for the second processed image by the recognition unit,
The image according to any one of appendix 1 to appendix 7, wherein the request unit requests the terminal device for an image corresponding to information that is insufficient in the recognition process for the second processed image. Recognition system.
(Appendix 9)
The terminal device and the server generate a processed image based on a procedure for requesting an image from the server to the terminal device until the recognition process by the recognition unit is completed, and the terminal device generates a processed image based on a request from the server. The interactive process including a procedure of transmitting to the server and a procedure of performing a recognition process on the processed image received by the server from the terminal device is performed. Image recognition system.
(Appendix 10)
The said server transmits the recognition result by the said recognition part to the said terminal device. The image recognition system as described in any one of Additional remarks 1-9 characterized by the above-mentioned.
(Appendix 11)
An image recognition device that receives and recognizes the processed image from a terminal device that generates a processed image from an input image,
A recognition unit that performs recognition processing on the processed image received from the terminal device;
A deficiency information determination unit for determining deficient information in the recognition processing by the recognition unit;
A request unit that requests an image corresponding to a determination result by the shortage information determination unit to the terminal device,
An image recognition apparatus comprising:
(Appendix 12)
An image recognition method for recognizing an image acquired by a terminal device in a server,
Receiving from the terminal device an initial image generated from the input image in the terminal device and having a smaller amount of information than the input image;
A recognition process is performed on the initial image,
Requesting the terminal device for an image corresponding to information lacking in the recognition process for the initial image;
An image generated in response to a request from the server in the terminal device is received from the terminal device,
An image recognition method, wherein the terminal device performs recognition processing on an image generated in response to a request from the server.

DESCRIPTION OF SYMBOLS 1 Terminal device 2 Server 13 Image processing part 14 Communication part 15 Request processing part 22 Recognition part 25 Insufficient information judgment part 26 Request part 31 Camera

Claims (6)

An image recognition system for recognizing an image acquired by a terminal device at a server,
The terminal device
An image processing unit for generating a processed image from the input image;
A communication unit that transmits the processed image generated by the image processing unit to the server,
The server
A recognition unit that performs recognition processing on the processed image received from the terminal device;
A deficiency information determination unit for determining deficient information in the recognition processing by the recognition unit;
A request unit for requesting an image corresponding to a determination result by the shortage information determination unit to the terminal device,
The image processing unit generates the processed image from the input image based on a request from the server. The image processing unit generates a first processed image having a smaller amount of information than the input image from the input image before the terminal device receives a request from the server,
The recognition unit performs recognition processing on the first processed image,
The insufficient information determination unit determines information that is insufficient in the recognition processing for the first processed image by the recognition unit,
The image recognition system according to claim 1, wherein the request unit requests the terminal device for an image corresponding to information that is insufficient in the recognition process for the first processed image. The image processing unit generates a second processed image from the input image based on a request from the server,
The recognition unit performs a recognition process on the second processed image,
The insufficient information determination unit determines information that is insufficient in the recognition processing for the second processed image by the recognition unit,
The image recognition system according to claim 1, wherein the request unit requests the terminal device for an image corresponding to information that is insufficient in the recognition process for the second processed image. The terminal device and the server generate a processed image based on a procedure for requesting an image from the server to the terminal device until the recognition process by the recognition unit is completed, and the terminal device generates a processed image based on a request from the server. The dialogue process including a procedure of transmitting to the server and a procedure of performing a recognition process on the processed image received by the server from the terminal device is performed. The image recognition system described. An image recognition device that receives and recognizes the processed image from a terminal device that generates a processed image from an input image,
A recognition unit that performs recognition processing on the processed image received from the terminal device;
A deficiency information determination unit for determining deficient information in the recognition processing by the recognition unit;
Based on the determination result by prior Symbol insufficient information determining unit, the image type, the range of the image, or the requesting unit reduction ratio to generate a request information specifying at least one transmission to the terminal device, the possess,
The recognizing unit receives a processed image generated according to the request information in the terminal device and executes a recognition process.
An image recognition apparatus characterized by that. An image recognition method for recognizing an image acquired by a terminal device in a server,
Receiving from the terminal device an initial image generated from the input image in the terminal device and having a smaller amount of information than the input image;
A recognition process is performed on the initial image,
Determining missing information in the recognition process for the initial image ;
Based on the determination result, request information specifying at least one of an image type, an image range, or a reduction ratio is generated and transmitted to the terminal device ,
A processed image generated according to the request information in the terminal device is received from the terminal device,
An image recognition method, comprising: performing recognition processing on the processed image received from the terminal device.