JP2015204037A - Image processing device, communication system, and communication method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of acquiring further information on an object by using a recognition result of an image while suppressing enlargement of the device and increase of communication time, and a communication system and a communication method using the image processing device.SOLUTION: An image processing device 100 includes: an image acquisition unit 102 for acquiring an image; an image recognition unit 104 for recognizing feature information from the image acquired by the image acquisition unit 102, the feature information being used for acquiring character information; a character information acquiring unit 106 for acquiring character information from the feature information recognized by the image recognition unit; and a communication unit 108 for communicating the character information acquired by the character information acquiring unit 106 to a guide information retrieving device 200 which retrieves the guide information from the character information.

Description

  The present invention relates to an image processing device, a communication system, and a communication method.

  In recent years, devices having an image recognition function have been proposed for devices such as mobile phones and digital cameras. For example, Patent Document 1 proposes a portable information reader that can read a barcode from an image obtained by imaging. Patent Document 2 proposes a code reading device capable of reading a two-dimensional code.

JP 2002-176509 A JP 2008-152569 A

  By recognizing the image, it is possible to obtain various types of feature information of the object shown in the image. If the feature information of the object is known, it is considered that further information of the object can be obtained by using the feature information. However, in order to obtain such additional information on the object by the image processing apparatus, it is necessary to store sufficient information about the object in the storage unit of the image processing apparatus. In this case, it is necessary to mount a large-capacity storage unit in the image processing apparatus. In addition, when an external device other than an image processing device such as a server is used to acquire further information about an image, it is necessary to communicate the image to the external device. Because of this image communication, the communication time tends to increase.

  The present invention has been made in view of the above circumstances, and is an image processing apparatus capable of obtaining further information on an object using an image recognition result while suppressing an increase in the size of the apparatus and an increase in communication time. It is another object of the present invention to provide a communication system and a communication method using such an image processing apparatus.

  In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention includes an image acquisition unit that acquires an image, and feature information that can acquire character information from the image acquired by the image acquisition unit. Retrieving guide information from character information for an image recognition unit to recognize, a character information acquisition unit to acquire character information from feature information recognized by the image recognition unit, and character information acquired from the character information acquisition unit And a communication unit that communicates with the guide information search device.

  In order to achieve the above object, an image processing apparatus according to a second aspect of the present invention includes an image acquisition unit that acquires an image, and feature information that can acquire character information from the image acquired by the image acquisition unit. An image recognition unit for recognizing and a communication unit for communicating the feature information recognized by the image recognition unit to a guide information search device that acquires character information from the feature information and searches for guide information. To do.

  In order to achieve the above object, a communication system according to a third aspect of the present invention recognizes an image acquisition unit that acquires an image, and feature information that can acquire character information from the image acquired by the image acquisition unit. An image recognition unit, a character information acquisition unit that acquires character information from the feature information recognized by the image recognition unit, and an image processing apparatus side communication unit that transmits the character information acquired by the character information acquisition unit. An image processing apparatus comprising: a guide information retrieval apparatus side communication unit that receives character information communicated from the image processing apparatus side communication unit; a search unit that retrieves guide information from the received character information; and the search And a guide information search device including a presentation unit for presenting the guide information to the user.

  In order to achieve the above object, a communication system according to a fourth aspect of the present invention recognizes an image acquisition unit that acquires an image, and feature information that can acquire character information from the image acquired by the image acquisition unit. Receiving the feature information communicated from the image processing device side communication unit, and the image processing device side communication unit that transmits the feature information acquired by the image recognition unit. A guide information search device side communication unit, a character information acquisition unit that acquires character information from the received feature information, a search unit that searches for guide information from the acquired character information, and the searched guide information And a guide information search device including a presentation unit to be presented to the user.

  In order to achieve the above object, a communication method according to a fifth aspect of the present invention includes obtaining an image, recognizing feature information capable of obtaining character information from the obtained image, and recognizing the recognition. Obtaining character information from the obtained feature information, communicating the obtained character information to a guide information retrieval device for retrieving guide information from the character information, and retrieving the guide information from the communicated character information. Searching for guide information in the apparatus, and presenting the guide information searched in the guide information searching apparatus to the user.

  In order to achieve the above object, a communication method according to a sixth aspect of the present invention includes obtaining an image, recognizing feature information capable of obtaining character information from the obtained image, and recognizing the recognition. Communicating the obtained feature information to a guide information retrieval device that obtains character information from the feature information and retrieves guide information, obtaining character information from the communicated feature information, and obtaining the obtained character Searching guide information from the information in the guide information search device, and presenting the guide information searched in the guide information search device to the user.

  ADVANTAGE OF THE INVENTION According to this invention, while suppressing the enlargement of an apparatus and the increase in communication time, the image processing apparatus which can obtain the further information of a target object using an image recognition result, and such an image processing apparatus are used. Communication system and communication method can be provided.

1 is a functional block diagram of a communication system including an image processing apparatus according to a first embodiment of the present invention. It is a figure which shows the structure of the communication system which is an application example of the image processing apparatus and guide information search device which concern on 1st Embodiment. It is a figure which shows the outline | summary of operation | movement of a communication system. 3 is a flowchart illustrating control of the imaging apparatus according to the first embodiment. It is the flowchart which showed control of the portable terminal in 1st Embodiment. FIG. 6A is a diagram illustrating an appearance of an imaging apparatus according to a second modification of the first embodiment, and FIG. 6B is a diagram illustrating an appearance of a portable terminal according to the second modification of the first embodiment. FIG. 6C is a diagram illustrating a mobile terminal to which an imaging device is attached via an attachment. It is the flowchart which showed control of the imaging device of the modification 2 of 1st Embodiment. It is FIG. 1 of the flowchart which showed control of the portable terminal of the modification 2 of 1st Embodiment. It is FIG. 2 of the flowchart which showed control of the portable terminal of the modification 2 of 1st Embodiment. It is a functional block diagram of the communication system containing the image processing apparatus which concerns on the 2nd Embodiment of this invention. It is a figure which shows the structure of the communication system which is an application example of the image processing apparatus and guide information search device which concern on 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
First, a first embodiment of the present invention will be described. FIG. 1 is a functional block diagram of a communication system including an image processing apparatus according to the first embodiment of the present invention. The image processing apparatus 100 includes an image acquisition unit 102, an image recognition unit 104, a character information acquisition unit 106, and a communication unit 108. The image processing apparatus 100 is configured to communicate with the guide information search apparatus 200.

  The image acquisition unit 102 acquires an image. The image acquisition unit 102 acquires an image obtained by imaging by an imaging unit, for example. In addition, the image acquisition unit 102 may be configured to acquire an image input from an external device of the image processing apparatus 100.

  The image recognition unit 104 recognizes the image input by the image acquisition unit 102. Here, “recognition” is a process of obtaining feature information that allows the character information obtaining unit 106 to obtain character information. For example, this recognition includes processing for recognizing the shape and color of an object in the image, processing for recognizing characters in the image, and the like. This recognition includes processing for recognizing information that can be quantified such as a distance to an object when the image is acquired, a position where the image is acquired, and a time when the image is acquired. The character information acquisition unit 106 acquires character information using the result recognized by the image recognition unit 104. For example, if the object recognition is performed in the image recognition unit 104, the character information acquisition unit 106 acquires character information such as the name of the object from the recognition result of the object.

  The communication unit 108 communicates with the guide information search device 200. For example, the communication unit 108 transmits the character information acquired by the character information acquisition unit 106 to the guide information search device 200. Further, the communication unit 108 receives information searched by the guide information search device 200.

  The guide information search apparatus 200 includes a storage unit 202, a search unit 204, a presentation unit 206, and a communication unit 208.

  The storage unit 202 stores character information and various types of information in association with each other. The various types of information associated with the character information are guide information that guides the user to a shooting method when shooting the object.

  The search unit 204 searches the storage unit 202 for information corresponding to the character information received by the communication unit 208. Note that the search unit 204 may be configured to acquire information from other than the storage unit 202. For example, the search unit 204 may acquire information by searching on a search site or the like using character information received by the communication unit 208 as a search keyword.

  The presentation unit 206 presents the information retrieved by the retrieval unit 204 to the user. This presentation is performed by display, for example.

  A communication unit 208 communicates with the image processing apparatus 100. For example, the communication unit 208 receives character information transmitted from the image processing apparatus 100. Further, the communication unit 208 transmits information searched by the search unit 204 to the image processing apparatus 100 as necessary.

  FIG. 2 is a diagram illustrating a configuration of a communication system that is an application example of the image processing apparatus 100 and the guide information search apparatus 200 illustrated in FIG. The communication system illustrated in FIG. 2 includes an imaging device 300, a mobile terminal 400, and a server 500. The imaging device 300 and the portable terminal 400 are configured to communicate with each other. Moreover, the portable terminal 400 and the server 500 are configured to communicate with each other.

  The imaging apparatus 300 has a function as the image processing apparatus 100, and includes a control unit 302, an imaging unit 304, a display unit 306, an eyepiece display unit 308, a recording unit 310, an operation unit 312, A touch panel 314, an attitude detection unit 316, a position detection unit 318, a timer unit 320, and a communication unit 322 are included. The imaging apparatus 300 is various devices having an imaging function such as a digital camera or a smartphone.

  The control unit 302 is configured by, for example, a CPU or an ASIC. The control unit 302 controls each unit of the imaging device 300. The control unit 302 includes the image acquisition unit 102, the image recognition unit 104, and the character information acquisition unit 106 of the image processing apparatus 100 illustrated in FIG. In addition, the control unit 302 includes an imaging control unit 3023, an image processing unit 3024, a display control unit 3025, and a communication control unit 3026. The imaging control unit 3023 controls the imaging operation by the imaging unit 304. The image processing unit 3024 performs image processing necessary for display and recording on the image acquired by the imaging unit 304. This image processing includes correction relating to image quality such as white balance correction, gradation correction, and color correction, resizing processing, compression processing, expansion processing, and the like. The display control unit 3025 controls display of various images on the display unit 306. The communication control unit 3026 performs control during communication between the imaging device 300 and an external device via the communication unit 322. By the communication control unit 3026, the imaging device 300 is connected to the portable terminal 400 so as to be communicable.

  The imaging unit 304 captures a subject and acquires an image (image data) related to the subject. The imaging unit 304 includes a photographic lens 3041, an imaging element 3042, and an analog / digital (A / D) conversion unit 3043. The taking lens 3041 condenses the light flux from the subject on the image sensor 3042. The imaging element 3042 has a light receiving surface. Pixels are arranged on the light receiving surface. The pixel is, for example, a photodiode and outputs an electrical signal (image signal) corresponding to the amount of incident light. The A / D conversion unit 3043 converts an analog image signal obtained by the imaging element 3042 into a digital image signal (image data).

  The display unit 306 is, for example, a liquid crystal display or an organic EL display, and is provided on the back surface of the imaging device 300, for example. The display unit 306 displays an image based on various image data. The eyepiece display unit 308 is, for example, a liquid crystal display or an organic EL display, and is, for example, a peeping type display unit provided on the back surface of the imaging apparatus 300. Similar to the display unit 306, the eyepiece display unit 308 displays an image based on various image data. Here, the image data used for display in the display unit 306 or the eyepiece display unit 308 is image data obtained by imaging by the imaging unit 304 or image data recorded in the recording unit 310.

  The recording unit 310 is a flash memory, for example, and records image data and the like as a file. In the recording unit 310, a connection destination database (DB) 3101 and an image database (DB) 3102 are constructed. A part of the recording area of the recording unit 310 is a model information recording unit 3103. The connection destination DB 3101 is a database that stores address information and the like necessary for communicating with a device (portable terminal 400) that is a communication target of the imaging apparatus 300. The image DB 3102 is a database that stores feature amount data of various objects necessary for image recognition by the image recognition unit 104. The model information recording unit 3103 records model information related to the imaging apparatus 300 that has captured an image for which guide information such as information on the model name of the imaging apparatus 300 and characteristic information (such as focal length) of the photographing lens 3041 is acquired. This is a recording area. Note that the recording unit 310 may be built in the imaging apparatus 300 or detachable from the imaging apparatus 300.

  The operation unit 312 is a mechanical operation member for the user to operate the imaging apparatus 300. Examples of the operation unit 312 include a release button and a power switch. The release button is a button for the user to instruct execution of the photographing operation. In addition, the power switch is a switch for the user to instruct to turn on or off the power of the imaging apparatus 300.

  The touch panel 314 is formed on the display screen of the display unit 306, for example, and detects a touch operation by the user. The control unit 302 performs processing according to the touch operation detected by the touch panel 314. For example, the touch panel 314 is used as a substitute for a release button. In addition, the touch panel 314 is used by the user to specify a specific portion of an image displayed on the display screen of the display unit 306.

  The posture detection unit 316 is a triaxial acceleration sensor, for example, and detects the posture of the imaging apparatus 300. The position detection unit 318 detects the current position of the imaging device 300 using, for example, GPS. Note that the position detection unit 318 may be configured to detect the orientation of the imaging device 300. The timer 320 acquires the current time. The communication unit 322 corresponds to the communication unit 108 and communicates with the mobile terminal 400 according to control by the communication control unit 3026. The communication unit 322 is, for example, a wireless LAN interface. The communication unit 322 may be a wired interface such as USB.

  The portable terminal 400 has a function as the guide information search apparatus 200 and includes a control unit 402, a recording unit 404, a display unit 406, a touch panel 408, a communication unit 410, and a communication unit 412. doing. The portable terminal 400 is various devices having a function of presenting information such as a smartphone. The portable terminal 400 in FIG. 2 does not have an imaging unit, but may have an imaging unit.

  The control unit 402 is constituted by a CPU, for example. The control unit 402 controls each unit of the mobile terminal 400. The control unit 402 includes the search unit 204 of the guide information search apparatus 200 shown in FIG. In addition, the control unit 402 includes a communication control unit 4022. The communication control unit 4022 performs control during communication with the imaging device 300 or the server 500.

  The recording unit 404 is, for example, a hard disk. In the recording unit 404, a guide information database (DB) 4041 as the storage unit 202 is constructed. The guide information DB 4041 is a database for searching guide information. The guide information is associated with various character information.

  The display unit 406 that functions as the presentation unit 206 is, for example, a liquid crystal display or an organic EL display. The display unit 406 displays an image based on various image data. The touch panel 408 is formed on the display screen of the display unit 406 and detects a touch operation by the user. The control unit 402 performs processing according to the touch operation detected by the touch panel 408.

  The communication unit 410 communicates with the imaging device 300 according to control by the communication control unit 4022. The communication unit 412 communicates with the server 500 according to control by the communication control unit 4022. The communication unit 410 is a wireless LAN interface, for example. The communication unit 412 is, for example, a mobile phone line.

  The server 500 includes a control unit 502, a recording unit 504, and a communication unit 506.

  The control unit 502 is configured by a CPU, for example, and controls each unit of the server 500. The control unit 502 includes a communication control unit 5021. The communication control unit 5021 performs control during communication with the mobile terminal 400. The recording unit 504 is, for example, a hard disk. The recording unit 504 stores various types of information searched by the search unit 204. The communication unit 506 communicates with the mobile terminal 400 according to control by the communication control unit 5021.

  Hereinafter, the operation of the communication system according to the present embodiment will be described. It is assumed that the user has the imaging device 300 and the portable terminal 400. In this state, the user points the imaging device 300 toward the subject as shown in FIG. 3A in order to photograph the subject (flowers in the figure). Here, it is assumed that the user is going to photograph cherry blossoms.

  At this time, the image recognition unit 104 of the imaging device 300 extracts feature information such as a subject from an image obtained via the imaging unit 304. The feature information is various types of information that can be converted into character information such as the shape and color of the subject and the distance from the imaging device 300 to the subject. The feature information is not limited to that acquired from the image. For example, when the position information is acquired by the position detection unit 318 or the current time is acquired by the time measuring unit 320, the current position information and the current time information may be used as the feature information. After extracting the feature information, the image recognition unit 104 transmits the extracted feature information to the character information acquisition unit 106. The character information acquisition unit 106 acquires character information from various feature information recognized by the image recognition unit 104. For example, in the example of FIG. 3B, “flower” that is character information representing the name of the subject and “pink” that is character information representing the color of the subject are extracted. In addition, “blue sky”, which is character information representing the background, is also extracted. If the information is digitized such as distance information, the numerical value is extracted as character information.

  After acquiring the character information, the communication control unit 3026 transmits the character information acquired by the character information acquisition unit 106 to the portable terminal 400 through the communication unit 322. At this time, the communication control unit 3026 may also transmit the model information recorded in the model information recording unit 3103 to the portable terminal 400 as character information.

  When the character information is acquired from the imaging device 300, the search unit 204 of the mobile terminal 400 searches the guide information DB 4041 using the character information acquired from the imaging device 300 as a search keyword as shown in FIG. To do. For example, if the name of the subject is “flower” and the model name of the imaging apparatus 300 is “XX-Z”, the search unit 204 “takes“ flower ”with“ XX-Z ””, “pink”, etc. Search with the search keyword. As described above, the search unit 204 may perform a search on the web through communication with the server 500.

  When the search for the guide information is completed, the control unit 402 of the mobile terminal 400 displays the searched guide information on the display unit 406 as shown in FIG. Thereby, the user can know how to take a flower using the model “XX-Z” as guide information. In the present embodiment, the imaging apparatus 300 only acquires character information for searching for guide information and transmits it to the mobile terminal 400, and does not search for guide information. Furthermore, the imaging apparatus 300 only transmits character information. Therefore, the communication load in the imaging apparatus 300 is considerably reduced.

  The details of the operation of the communication system of the present embodiment will be described below. FIG. 4 is a flowchart showing the control of the imaging apparatus 300. The process of FIG. 4 is performed according to a program stored in the control unit 302 in advance.

  In FIG. 4, the control unit 302 determines whether or not the power switch is turned on (step S101). If it is determined in step S101 that the power switch is not turned on, the control unit 302 causes the imaging apparatus 300 to transition to the standby state. In the standby state, only the minimum necessary functions in the imaging apparatus 300 such as the determination in step S101 are executed.

  When it is determined in step S101 that the power switch is turned on, the control unit 302 determines whether or not the operation mode of the imaging apparatus 300 is the shooting mode (step S102). The shooting mode is an operation mode for shooting a recording image. Note that the operation mode of the imaging apparatus 300 is determined by, for example, an operation of the operation unit 312 or an operation of the touch panel 314 by the user. If it is determined in step S102 that the operation mode of the imaging apparatus 300 is not the shooting mode, the control unit 302 performs processing according to an operation mode other than the shooting mode. An operation mode other than the shooting mode is, for example, a playback mode. Description of the details of the playback mode is omitted.

  If it is determined in step S102 that the operation mode of the imaging apparatus 300 is the shooting mode, the control unit 302 performs a through image display (step S103). The through image display is an operation in which an image obtained by the continuous operation of the imaging unit 304 is displayed on the display unit 306 in real time. When displaying a through image, the control unit 302 acquires an image obtained by the imaging unit 304 during the through image display by the image acquisition unit 102. Then, the control unit 302 causes the image recognition unit 104 to extract feature information from the image acquired by the image acquisition unit 102 (step S104).

  Subsequently, the control unit 302 determines whether or not the user requests guide information (step S105). For example, when the guide information is requested by the operation of the operation unit 312 or the touch panel 314 by the user, it is determined that the user is requesting the guide information. When it is determined in step S105 that the user does not request guide information, the control unit 302 skips the processes in steps S106 to S109. On the other hand, if it is determined in step S105 that the user requests guide information, the control unit 302 acquires character information from the feature information extracted by the image recognition unit 104 by the character information acquisition unit 106 (step S105). S106). Thereafter, the control unit 302 determines whether character information has been acquired (step S107). When it is determined in step S107 that character information could not be acquired, the control unit 302 issues a warning to the user that character information could not be acquired, that is, guide information could not be acquired (step S108). . The warning is performed using, for example, display or sound. After the warning, the control unit 302 shifts the process to step S110.

  When it is determined in step S107 that the character information has been acquired, the control unit 302 transmits the character information acquired by the character information acquisition unit 106 to the portable terminal 400 by the communication control unit 3026 (step S109). Of course, an application in which only information that can be converted into characters, such as color, shape, size, and distance distribution, is sent as image information, and the image information itself is sent as auxiliary information. In this case, the image information may be only the outline or only the color distribution. The time, place, temperature, humidity, atmospheric pressure, direction, inclination, etc. obtained in an auxiliary manner may be sent as text information. For example, if the temperature information is converted to “when it is hot” or the elevation angle information is converted to “high in the sky” and sent, various text searches, net searches, and natural language searches are facilitated. Texting is also important for shooting modes, etc. From the name of the mode and shooting parameters, if you send characters such as “Shoot with Macro” or “Shoot with Wide” in text, the amount of information can be increased and the information can be narrowed down. It is easy to search for shooting results by shooting method.

  If it is determined in step S105 that the user does not request guide information, after step S108 or S109, the control unit 302 determines whether there is a photographing operation by the user (step S110). The photographing operation is, for example, a release button operation by the user or a touch release operation using the touch panel 314. If it is determined in step S110 that there is no shooting operation by the user, the control unit 302 returns the process to step S101.

  When it is determined in step S110 that there is a shooting operation by the user, the control unit 302 performs a shooting operation (step S111). The shooting operation is an operation for controlling the imaging unit 304 to acquire image data for recording. After executing the photographing operation, the control unit 302 causes the image processing unit 3024 to perform image processing on the image data obtained by the photographing operation. Then, the control unit 302 records the image data after image processing in the recording unit 310 (step S112). Thereafter, the control unit 302 returns the process to step S101.

  FIG. 5 is a flowchart showing control of the mobile terminal 400. The process of FIG. 5 is performed according to a program stored in the control unit 402 in advance.

  In FIG. 5, the control unit 402 determines whether or not to turn on the power (step S201). When it is determined in step S201 that the power is not turned on, the control unit 402 shifts the portable terminal 400 to the standby state. In the standby state, only the minimum necessary functions in the portable terminal 400 such as the determination in step S201 are executed.

  When it is determined in step S201 that the power is to be turned on, the control unit 402 turns on the mobile terminal 400. Then, the control unit 402 determines whether communication with the imaging apparatus 300 is in progress (step S202). If it is determined in step S202 that communication with the imaging apparatus 300 is not in progress, the control unit 402 performs processing according to an operation mode other than the communication mode with the imaging apparatus 300. The details of the operation modes other than the communication mode will not be described.

  If it is determined in step S202 that communication with the imaging apparatus 300 is in progress, the control unit 402 determines whether character information has been received from the imaging apparatus 300 (step S203). When determining in step S203 that no character information has been received, the control unit 402 returns the process to step S201.

  When it is determined in step S203 that character information has been received, the control unit 402 causes the search unit 204 to search for guide information using the received character information (step S204). As described above, the guide information can be searched not only from the guide information DB 4041 but also on the web by the server 500. After retrieving the guide information, the control unit 402 determines whether or not the guide information has been retrieved (step S205). If it is determined in step S205 that the guide information cannot be searched, the control unit 402 issues a warning that the guide information cannot be acquired (step S206). The warning is performed using, for example, display or sound. After the warning, the control unit 402 shifts the process to step S201.

When it is determined in step S205 that the guide information can be searched, the control unit 402 displays the searched guide information on the display unit 406 as shown in FIG. 3D, for example (step S207). This guide information may be advice at the time of shooting or observation, not only the name and explanation of the object obtained at the time of shooting and observation, but also the things behind it, or the historical background It may be a guide to a relationship with a work of art. In addition to displaying here, it may be displayed by a camera, and when there is a related device in the vicinity or when there is a related device even remotely, it may be wirelessly skipped so as to be displayed there. For example, it may be useful for a meeting or the like if it is adopted in a system that displays information related to the image taken at that time together with an image taken at a business trip on a large monitor set up in the workplace. The bride's bouquet taken at the time of shooting at the wedding hall is projected onto the projector.
After displaying the guide information, the control unit 402 determines whether or not to end the display of the guide information (step S208). For example, when the end of the display of the guide information is requested by the operation of the touch panel 408 by the user, it is determined that the display of the guide information is ended. If it is determined in step S208 that the display of the guide information is not to be ended, the control unit 402 returns the process to step S207 to continue the display of the guide information. If it is determined in step S208 that the display of the guide information is to be terminated, the control unit 402 returns the process to step S201.

  As described above, according to the present embodiment, it is possible to acquire guide information using an image recognition result. Here, in this embodiment, guide information is acquired by the guide information search apparatus 200. For this reason, it is not necessary to store information for acquiring guide information in the image processing apparatus 100. Further, communication that occurs in the image processing apparatus 100 is only communication when character information is transmitted to the portable terminal 400. For this reason, it is possible to reduce the communication load in the image processing apparatus 100.

Next, a modification of this embodiment will be described.
[Modification 1]
In the embodiment described above, guide information is displayed on the display unit 406 of the portable terminal 400 as the guide information search device 200. On the other hand, although communication with the imaging device 300 as the image processing device 100 occurs, the guide information acquired in the portable terminal 400 may be displayed on the display unit 306 or the eyepiece display unit 308 of the imaging device 300. . In particular, when guide information is displayed on the eyepiece display unit 308, the displayed guide information is not affected by external light. For this reason, the user can see the guide information in a clear state.

[Modification 2]
FIG. 3 shows an example in which the user has the imaging device 300 and the portable terminal 400 separately. On the other hand, the imaging device 300 and the portable terminal 400 may be used as shown in FIG.

  FIG. 6A shows the appearance of the imaging apparatus 300 of the second modification. An imaging apparatus 300 illustrated in FIG. 6A is an “interchangeable lens-shaped” imaging apparatus. Each processing block of the imaging apparatus 300 shown in FIG. 2 is provided inside an interchangeable lens-shaped casing. Note that in the case of the interchangeable-lens imaging device 300, it is not necessary to provide all the processing blocks shown in FIG. For example, the display unit 306, the eyepiece display unit 308, the touch panel 314, the posture detection unit 316, the position detection unit 318, and the time measuring unit 320 may not be present (may be present). On the other hand, FIG. 6B shows the appearance of the mobile terminal 400. A mobile terminal 400 of Modification 2 is a smartphone, for example, and has the same configuration as the mobile terminal 400 shown in FIG.

  As illustrated in FIG. 6C, the imaging apparatus 300 according to Modification 2 is attached to the mobile terminal 400 via the attachment 600. In such a configuration, an image captured by the imaging apparatus 300 is transmitted to the mobile terminal 400. That is, the imaging device 300 and the mobile terminal operate like a single digital camera.

  FIG. 7 is a flowchart illustrating control of the imaging apparatus 300 according to the second modification. The processing in FIG. 7 is performed according to a program stored in advance in the control unit 302.

  In FIG. 7, the control unit 302 determines whether or not the power is turned on (step S301). For example, when the imaging apparatus 300 is provided with a power switch, it is determined whether or not the power switch is turned on. If it is determined in step S301 that the power is not turned on, the control unit 302 causes the imaging apparatus 300 to transition to the standby state.

  If it is determined in step S301 that the power is turned on, the control unit 302 causes the communication control unit 3026 to transmit a through image to the portable terminal 400 (step S302). At this time, the control unit 302 acquires a through image obtained by the imaging unit 304 by the image acquisition unit 102. Then, the control unit 302 causes the image recognition unit 104 to extract feature information from the image acquired by the image acquisition unit 102 (step S303).

  Subsequently, the control unit 302 determines whether or not the user requests guide information (step S304). For example, when a guide request from the mobile terminal 400 is received, it is determined that the user is requesting guide information. If it is determined in step S304 that the user does not request guide information, the control unit 302 skips the processes in steps S305 to S308. On the other hand, when it is determined in step S304 that the user is requesting guide information, the control unit 302 acquires character information from the feature information extracted by the image recognition unit 104 by the character information acquisition unit 106 (step S304). S305). Thereafter, the control unit 302 determines whether or not the character information has been acquired (step S306). When it is determined in step S306 that character information could not be acquired, the control unit 302 causes the communication control unit 3026 to transmit warning information indicating that character information could not be acquired to the portable terminal 400 (step S307). After transmission of the warning information, the control unit 402 shifts the process to step S309. However, the acquisition of text information is not so difficult, and it may be converted into “spring” or “early summer” as seasonal information from the time of shooting, such as “early morning” or “in the evening”. Characters can be converted immediately. Only a color such as “yellow” or a shape such as “round” may be used. “Bright” or “moving” can be said to be information that can be easily obtained by shooting. In many cases, distance information can also be obtained at the time of shooting, and such change with time may be used as image information.

  When it is determined in step S306 that the character information has been acquired, the control unit 302 transmits the character information acquired by the character information acquisition unit 106 to the portable terminal 400 by the communication control unit 3026 (step S308).

  If it is determined in step S304 that the user does not request guide information, after step S307 or S308, the control unit 302 determines whether there is a shooting instruction from the portable terminal 400 (step S309). If it is determined in step S309 that there is no shooting instruction, the control unit 302 returns the process to step S301.

  When it is determined in step S309 that there is a shooting instruction, the control unit 302 performs a shooting operation (step S310). After executing the photographing operation, the control unit 302 causes the image processing unit 3024 to perform image processing on the image data obtained by the photographing operation. And the control part 302 transmits the image data after an image process to the portable terminal 400 by the communication control part 3026 (step S311). Thereafter, the control unit 302 returns the process to step S301.

  8A and 8B are flowcharts illustrating control of the mobile terminal 400 according to the second modification. The processes in FIGS. 8A and 8B are performed according to a program stored in advance in the control unit 402.

  In FIG. 8A, the control unit 402 determines whether to turn on the power (step S401). If it is determined in step S401 that the power is not turned on, the control unit 402 causes the portable terminal 400 to transition to the standby state.

  When it is determined in step S401 that the power is to be turned on, the control unit 402 turns on the mobile terminal 400. Then, the control unit 402 determines whether or not the operation mode of the mobile terminal 400 is the shooting mode (step S402). The operation mode is set by, for example, the operation of the touch panel 408 by the user. When it is determined in step S402 that the operation mode is not the shooting mode, the control unit 402 performs processing according to an operation mode other than the shooting mode. The details of the operation modes other than the shooting mode will not be described.

  When it is determined in step S402 that the operation mode is the shooting mode, the control unit 402 determines whether communication with the imaging apparatus 300 is in progress (step S403). If it is determined in step S403 that communication with the imaging device 300 is not in progress, the control unit 402 warns the user that the mobile terminal 400 and the imaging device 300 are not in communication (step S404). Thereafter, the control unit 402 returns the process to step S401.

  If it is determined in step S403 that communication is being performed with the imaging apparatus 300, the control unit 402 determines whether a through image has been received from the imaging apparatus 300 (step S405). If it is determined in step S405 that no through image has been received, the control unit 402 returns the process to step S401. When determining in step S405 that a through image has been received, the control unit 402 causes the display unit 406 to display the received through image (step S406).

  Subsequently, the control unit 402 determines whether or not the user requests guide information (step S407). For example, when the guide information is requested by the operation of the touch panel 408 by the user, it is determined that the user is requesting the guide information. When it is determined in step S407 that the user does not request guide information, the control unit 402 skips the processes in steps S408 to S414. On the other hand, when it is determined in step S407 that the user is requesting guide information, the control unit 402 transmits a guide request to the imaging apparatus 300 through the communication control unit 4022 (step S408).

  After transmitting the guide request, the control unit 402 determines whether character information has been received from the imaging apparatus 300 (step S409). If it is determined in step S409 that no character information has been received, the control unit 402 determines whether warning information has been received from the imaging apparatus 300 (step S410). If it is determined in step S410 that warning information has not been received, the control unit 402 returns the process to step S409 and waits for reception of character information. If it is determined in step S410 that the warning information has been received, the control unit 402 warns the user that the guide information has not been acquired (step S411). The warning is performed using, for example, display or sound. After the warning, the control unit 402 shifts the process to step S415.

  If it is determined in step S409 that character information has been received, the control unit 402 causes the search unit 204 to search for guide information using the received character information (step S412). The search for guide information in step S412 is performed in the same manner as the search for guide information in step S204. After retrieving the guide information, the control unit 402 determines whether or not the guide information has been retrieved (step S413). If it is determined in step S413 that the guide information could not be searched, the control unit 402 shifts the process to step S411 and issues a warning.

  If it is determined in step S413 that the guide information has been searched, the control unit 402 causes the display unit 406 to display the searched guide information as shown in FIG. 3D (step S414).

  If it is determined in step S407 that the user does not request guide information, after step S411 or S414, the control unit 402 determines whether there is a photographing operation by the user (step S415). The photographing operation is, for example, a touch release operation using the touch panel 408 by the user. When determining in step S415 that there is no shooting operation by the user, the control unit 402 returns the process to step S401.

  When it is determined in step S415 that there is a shooting operation by the user, the control unit 302 transmits a shooting instruction to the imaging apparatus 300 through the communication control unit 4022 (step S416). After transmitting the shooting instruction, the control unit 402 determines whether image data has been received from the imaging device 300 (step S417). The control unit 402 repeats the determination in step S417 until it is determined in step S417 that image data has been received from the imaging apparatus 300. If image data cannot be received within a predetermined time, the process may proceed to step S401.

  When determining in step S417 that an image has been received from the imaging apparatus 300, the control unit 402 records the received image data in the recording unit 404 (step S418). Thereafter, the control unit 402 returns the process to step S401.

  As described above, even with the configuration of the second modification, the same effects as those of the above-described embodiment can be obtained. In addition, the imaging apparatus 300 can increase the degree of freedom in designing an imaging element and a photographing lens as compared with the portable terminal 400. For this reason, it is possible to provide an imaging unit with higher functionality than the portable terminal 400. By providing a high-function image capturing unit, the image capturing apparatus 300 can capture a higher quality image than the portable terminal 400. Further, by transmitting such a high-quality image to the portable terminal 400, a high-quality through image is displayed on the display unit 406 of the portable terminal 400, or a high-quality image is recorded on the recording unit 404. It is possible.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. FIG. 9 is a functional block diagram of a communication system including an image processing apparatus according to the second embodiment of the present invention. FIG. 10 is a diagram illustrating a configuration of a communication system that is an application example of the image processing apparatus 100 and the guide information search apparatus 200 illustrated in FIG. 9. Only parts different from those of the first embodiment will be described with reference to FIGS. As shown in FIGS. 9 and 10, in the second embodiment, the character information acquisition unit 106 is provided in the guide information search device 200 (mobile terminal 400). Other points are the same as in FIG. In such a configuration, in the image processing apparatus 100 of the second embodiment, the object recognition by the image recognition unit 104 is performed. In the guide information search apparatus 200, the character information is acquired and the guide information is searched.

  Also in the present embodiment as described above, it is possible to acquire guide information using an image recognition result. Further, communication that occurs in the image processing apparatus 100 is only communication when feature information is transmitted to the guide information search apparatus 200. For this reason, it is possible to reduce the communication load in the image processing apparatus 100.

  Note that the configuration described in the second embodiment and the configurations of the first and second modifications of the first embodiment may be combined.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention. Further, in the description of each operation flowchart described above, the operation is described using “first”, “next”, and the like for convenience, but this does not mean that it is essential to perform the operations in this order. Absent.

  Further, in the above-described embodiment, the aspect as the image processing apparatus has been emphasized. However, the technology of the above-described embodiment can be used as an image display device, an inspection device, and a warning device, for example, industrial equipment and medical devices other than consumer applications. It can also be applied to the field of industrial equipment. The richness of information amount at the time of text-based search and the information narrowing down information can improve the accuracy at the time of display, inspection and warning. Even if you don't have a mobile communication terminal such as a smartphone, you can make specifications so that auxiliary information is sent to you. If the inspection is not successful, the failure inspection image may be judged from the image or information at the time of shooting, and may be sent as “failure”. Such applications are within the scope covered by the technology of the present application. Yes, if the terminal is taken out, it is convenient if advice to prevent failure is displayed.

  Further, the above-described embodiments include various stages of the invention, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some configuration requirements are deleted from all the configuration requirements shown in the embodiment, the above-described problem can be solved, and this configuration requirement is deleted when the above-described effects can be obtained. The configuration can also be extracted as an invention.

  DESCRIPTION OF SYMBOLS 100 Image processing apparatus, 102 Image acquisition part, 104 Image recognition part, 106 Character information acquisition part, 108 Communication part, 200 Guide information search device, 202 Storage part, 204 Search part, 206 Presentation part, 208 Communication part, 300 Imaging device , 302 control unit, 304 imaging unit, 306 display unit, 308 eyepiece display unit, 310 recording unit, 312 operation unit, 314 touch panel, 316 posture detection unit, 318 position detection unit, 320 timing unit, 322 communication unit, 400 portable terminal , 402 control unit, 404 recording unit, 406 display unit, 408 touch panel, 410 communication unit, 412 communication unit, 500 server, 502 control unit, 504 recording unit, 506 communication unit, 600 attachment, 3023 imaging control unit, 3024 image processing Part, 3025 Display control part, 3026 Communication control unit, 3041 photographing lens, 3042 imaging device, 3043 digital (A / D) conversion unit, 3103 model information recording unit, 4022 communication control unit, 5021 communication control unit, 3101 connection destination database (DB), 3102 image database ( DB), 4041 Guide information database (DB)

Claims (9)

An image acquisition unit for acquiring images;
An image recognition unit for recognizing feature information capable of acquiring character information from the image acquired by the image acquisition unit;
A character information acquisition unit that acquires character information from the feature information recognized by the image recognition unit;
A communication unit that communicates the character information acquired by the character information acquisition unit to a guide information search device that searches for guide information from the character information;
An image processing apparatus comprising: An image acquisition unit for acquiring images;
An image recognition unit for recognizing feature information capable of acquiring character information from the image acquired by the image acquisition unit;
A communication unit that communicates to the guide information search device that acquires the character information from the feature information and retrieves the guide information, the feature information recognized by the image recognition unit;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the information from which the character information can be acquired includes information on a shape and color of an object in the image and distance information to the object. A model information recording unit that records model information of the imaging apparatus that has acquired the image;
The image processing apparatus according to claim 3, wherein the communication unit further communicates the model information to the guide information search apparatus. A position detection unit for detecting a position where the image is acquired;
A timer for detecting the time when the image was acquired;
Further comprising
The image processing apparatus according to claim 3, wherein the image recognition unit further recognizes the position and time as the feature information. An image acquisition unit for acquiring images;
An image recognition unit for recognizing feature information capable of acquiring character information from the image acquired by the image acquisition unit;
A character information acquisition unit that acquires character information from the feature information recognized by the image recognition unit;
An image processing apparatus-side communication unit that transmits the character information acquired by the character information acquisition unit;
An image processing apparatus comprising:
A guide information retrieval device side communication unit that receives character information communicated from the image processing device side communication unit;
A search unit for searching for guide information from the received character information;
A presentation unit for presenting the searched guide information to a user;
A guide information retrieval device comprising:
A communication system comprising: An image acquisition unit for acquiring images;
An image recognition unit for recognizing feature information capable of acquiring character information from the image acquired by the image acquisition unit;
An image processing device side communication unit for transmitting the feature information acquired by the image recognition unit;
An image processing apparatus comprising:
A guide information retrieval device side communication unit that receives feature information communicated from the image processing device side communication unit;
A character information acquisition unit for acquiring character information from the received feature information;
A search unit for searching for guide information from the acquired character information;
A presentation unit for presenting the searched guide information to a user;
A guide information retrieval device comprising:
A communication system comprising: Acquiring images,
Recognizing feature information capable of acquiring character information from the acquired image;
Obtaining character information from the recognized feature information;
Communicating the acquired character information to a guide information retrieval device that retrieves guide information from the character information;
Retrieving guide information in the guide information retrieval device from the communicated character information;
Presenting guide information retrieved by the guide information retrieval device to the user;
A communication method comprising: Acquiring images,
Recognizing feature information capable of acquiring character information from the acquired image;
Communicating the recognized feature information to a guide information retrieval device that retrieves character information from the feature information and retrieves guide information;
Obtaining character information from the communicated feature information;
Searching the guide information in the guide information search device from the acquired character information;
Presenting guide information retrieved by the guide information retrieval device to the user;
A communication method comprising:

Images