JP2013130919A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To safely use an augmented reality function.SOLUTION: A CPU 11 of an imaging device which executes pattern information processing, comprises a captured image acquisition unit 45, a pattern information extraction unit 47, a pattern information determination unit 48, and a presentation information acquisition unit 49. The captured image acquisition unit 45 acquires data of an image including a subject which is represented by a pattern and presents information at the same time. The pattern information extraction unit 47 identifies the subject in the image acquired by the captured image acquisition unit 45. The pattern information determination unit 48 examines whether the pattern representing the subject identified by the pattern information extraction unit 47 is safe. The presentation information acquisition unit 49 acquires information presented by the subject when it is determined that the pattern representing the subject is safe on the basis of the examination result by the pattern information determination unit 48.

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program that can safely use augmented reality.

In recent years, so-called augmented reality technology that combines and presents virtual objects as additional information in a part of an actual environment has attracted attention in many fields. In particular, game applications for smartphones and the like often use augmented reality technology such as displaying a virtual object on a smartphone screen using a camera function mounted on the smartphone.
As a technique for realizing augmented reality, for example, a technique described in Patent Document 1 is known. In the technique described in Patent Document 1, a marker loaded on a subject to be imaged is recognized, and CG (Computer Graphics) is automatically included in a captured image of a virtual object based on information of the recognized marker. Augmented reality is realized by overwriting and displaying.

Special table 2008-510254 gazette

In augmented reality, as an orthodox technique, there is a technique in which a CG is automatically overwritten and displayed in an image by a recognized marker as in the technique described in Patent Document 1 described above. Depending on the information, various operations can be automatically executed. That is, depending on the information of the marker, for example, a device that executes augmented reality can be operated in any manner.
A user who has executed augmented reality that automatically connects to a specific server or automatically executes a code containing a virus by using a marker created by a malicious person or a marker with a bug or other defect. May be configured to result in disadvantages.
However, at present, there is no way to deal with markers created by such a malicious person or markers with defects such as bugs, and users using augmented reality may be exposed to unexpected danger. is there.

  Therefore, the present invention has been made in view of such a situation, and an object thereof is to provide an information processing apparatus, an information processing method, and a program that can safely use an augmented reality function.

  In order to achieve the above object, an imaging device of one embodiment of the present invention includes an image acquisition unit that acquires data of an image that is represented by a pattern and includes a subject that presents information, and an image acquired by the image acquisition unit Identification means for identifying the subject from, a first inspection means for inspecting whether or not the pattern representing the subject identified by the identification means is safe, and the first inspection means And an information acquisition unit configured to acquire information presented by the subject when the pattern representing the subject is determined to be safe by the examination performed.

  According to the present invention, an augmented reality function can be used safely.

It is a schematic diagram which shows the example at the time of reading the marker of augmented reality and performing with an imaging device. It is a schematic diagram which shows the other example at the time of reading the marker of augmented reality and performing with an imaging device. It is a block diagram which shows the hardware constitutions of the imaging device which concerns on one Embodiment of the information processing apparatus of this invention. It is a functional block diagram which shows the functional structure for performing a pattern information confirmation process among the functional structures of the imaging device of FIG. 5 is a flowchart for explaining a flow of pattern information confirmation processing executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 4. It is a flowchart explaining the detailed flow of a process when modes other than an augmented reality mode are set among the pattern information confirmation processes of FIG. It is a flowchart explaining the flow of the light-emitting body information confirmation process which the imaging device in this embodiment performs. It is a figure which shows the modification in this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, the principle of augmented reality will be described with reference to FIG.
FIG. 1 is a schematic diagram illustrating an example in which an augmented reality marker is read and executed by an imaging apparatus.
In the example of FIG. 1, a user uses an imaging device to separate a road on which a vehicle passes, a plurality of street trees planted along the road, and a landscape in which a plurality of buildings are built behind the road tree. Shooting.
In the imaging range, a marker that exhibits an augmented reality function (two-dimensional in the example of FIG. 1) is provided on the wall surface of one of the buildings that are the subject (the rightmost building in the example of FIG. 1). Marker expressed by code) is drawn.
When a user captures a landscape including a building on which a marker is drawn, the landscape is displayed as a captured image on the screen of display means provided in the imaging device.
At that time, processing related to augmented reality is executed on the screen based on the information of the marker drawn on the wall surface of the building.
As a result of the processing related to augmented reality, an image that does not actually exist (in the example of FIG. 1, a CG image of a smile mark) is displayed on the screen at the position corresponding to the marker along with the actual landscape. Is done.
As described above, in the imaging apparatus, the function corresponding to the marker is executed on the basis of the marker in the reality by performing the augmented reality function. In other words, in the example of FIG. 1, a process is performed in which an image is displayed corresponding to the marker, instead of the actually drawn marker.

Next, another example of augmented reality different from the example of FIG. 1 will be described.
FIG. 2 is a schematic diagram illustrating another example in which an augmented reality marker is read and executed by the imaging apparatus.

The example of FIG. 2 is an example in which a landscape around the office desk is photographed by an imaging device, and a marker that performs an augmented reality function is displayed on the screen of the notebook computer on the desk.
Unlike the example of FIG. 1, the example of FIG. 2 is common in that an augmented reality function is exhibited in the imaging device by imaging a landscape around the office desk, but is stored in the imaging device. A process of transferring data to another device is executed.
When this augmented reality function is exhibited, the user unintentionally transfers the data stored in the imaging device to another device.
For this reason, if an augmented reality function such as this example is used and a malicious person places the marker in various places and places the marker in the imaging range, the user's intention Regardless of the data, the data in the imaging apparatus can be leaked. That is, it becomes possible to abuse the augmented reality function.

  In terms of misuse, the example of FIG. 1 also exploits augmented reality functions, for example, by creating a marker that operates to display harmful contents of the image to be displayed. Is possible.

  It is difficult to find a marker that exhibits an augmented reality function, and it is difficult to determine the function that is performed even when the marker itself is viewed, that is, the processing content to be executed.

Augmented reality functions are more likely to be used depending on the use, and may be applied to various fields in the future. However, as usage increases, more people will abuse this function as described above. There is concern.
Moreover, not only a marker intentionally created under malicious intentions, but also when a defective marker was created, it was intentionally created under malicious intentions that the creator himself did not intend A case where processing that adversely affects a person who uses an augmented reality function in the same manner as the marker is executed is also conceivable.

  Therefore, the present invention provides an information processing apparatus that has a function of automatically determining a marker that adversely affects a person who uses the augmented reality function as described above and information acquired based on the marker.

An imaging apparatus according to an embodiment of the information processing apparatus of the present invention having the functions as described above will be described with reference to FIG.
FIG. 3 is a block diagram showing a hardware configuration of the imaging apparatus 1 according to an embodiment of the information processing apparatus of the present invention.
The imaging device 1 is configured as a digital camera, for example.

  The imaging apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an imaging unit 16, and an input unit 17. An output unit 18, a storage unit 19, a communication unit 20, and a drive 21.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

  The CPU 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output interface 15 is also connected to the bus 14. An imaging unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, and a drive 21 are connected to the input / output interface 15.

  Although not shown, the imaging unit 16 includes an optical lens unit and an image sensor.

The optical lens unit is configured by a lens that collects light, for example, a focus lens or a zoom lens, in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor includes a photoelectric conversion element, AFE (Analog Front End), and the like.
The photoelectric conversion element includes, for example, a CCD (Charge Coupled Device Image Sensor) type or a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. Through various signal processing, a digital signal is generated and output as an output signal of the imaging unit 16.
Hereinafter, the output signal of the imaging unit 16 is referred to as “captured image data”. The captured image data is appropriately supplied to the CPU 11 or the like.

  The imaging unit 16 is configured to receive visible light. By recognizing the variable state such as high-speed blinking of the received visible light by the imaging unit 16, the imaging device 1 can receive visible light communication, that is, a signal of visible light communication from the outside. It becomes.

The input unit 17 is configured with various buttons and the like, and inputs various information according to a user's instruction operation.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 20 controls communication performed with other devices (not shown) via a network including the Internet.

  A removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 21. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

In addition, the pattern in this application is a spatial pattern or a temporal pattern.
Next, a functional configuration for executing the pattern information confirmation process among the functional configurations of the imaging apparatus 1 will be described with reference to FIG.
FIG. 4 is a functional block diagram showing a functional configuration for executing the pattern information confirmation process among the functional configurations of the imaging apparatus 1 of FIG.
Here, the pattern information confirmation process is a series of processes for confirming the safety of information acquired from the captured marker and executing the information content whose safety is confirmed.

  When the imaging apparatus 1 executes the pattern information confirmation process, as shown in FIG. 4, in the CPU 11, the main control unit 41, the input detection unit 42, the mode setting unit 43, the imaging control unit 44, and the imaging The image acquisition unit 45, the output control unit 46, the pattern information extraction unit 47, the pattern information determination unit 48, the presentation information acquisition unit 49, the presentation information determination unit 50, and the presentation information content execution unit 51 function. .

  Further, a pattern information table storage unit 71 and a presentation information table storage unit 72 are provided as one area of the storage unit 19 of the imaging device 1. The pattern information table storage unit 71 and the presentation information table storage unit 72 are configured as one area of the storage unit 19, but may be provided as another area of the removable media 31, for example.

The pattern information table storage unit 71 stores a pattern information table that is information related to pattern information indicating a marker pattern. The pattern information table is a table in which pattern information and information on safety evaluation of whether the pattern information is safe or dangerous are associated with each other and stored.
In the case of the example of FIG. 1, the pattern information table is configured as a table in which various patterns of the two-dimensional code and the safety evaluation result for each pattern are stored in association with each other.

The presentation information table storage unit 72 stores a presentation information table that is information related to the presentation information presented based on the marker. The presentation information table is a table in which presentation information and information on whether the presentation information is safe or dangerous are evaluated and associated with each other.
In the case of the example of FIG. 1, the pattern information table is configured as a table in which an acquired image and a safety evaluation result for each image are associated and stored.

The main control unit 41 controls the overall execution of various processes (in this embodiment, processes such as an augmented reality mode, an imaging mode, a visible light communication mode, or other modes (such as a playback mode)).
The input detection unit 42 detects the content of the operation performed on the input unit 17 by the user.
The mode setting unit 43 is a mode selected from a plurality of modes (augmented reality mode, imaging mode, visible light communication mode, or other modes (reproduction mode, etc.)) based on selection by the user operation to the input unit 17 The setting conditions corresponding to are automatically set. Here, the augmented reality mode is a mode in which the augmented reality function is exhibited by the marker recognized in the shooting state. The imaging mode is a mode for saving a captured image in a shooting state. The visible light communication mode is a mode in which various processes are executed using visible light (illuminant) recognized in the imaging state as a communication signal.

  The imaging control unit 44 executes control related to imaging in the imaging unit 16 such as imaging, AF (Auto Focus), AE (Automatic Exposure), AWB (Auto White Balance), and the like.

  The captured image acquisition unit 45 acquires captured image data from the imaging unit 16. The captured image acquisition unit 45 supplies the output unit 18 for live view display and the storage unit 19 for recording.

  The output control unit 46 executes output control in the output unit 18. For example, the output control unit 46 controls display output of an image or the like in the output unit 18.

  The position of a predetermined shape is recognized from the pattern information extraction unit 47 and the captured image acquired by the captured image acquisition unit 45, the pattern at the recognized position is recognized as a marker, and the pattern information is extracted from the marker. In the example of FIG. 1, the marker of the two-dimensional code drawn on the wall surface of the building is recognized, and the design of the marker is extracted as pattern information from the recognized marker of the two-dimensional code.

The pattern information determination unit 48 determines the pattern information acquired by the pattern information extraction unit 47. Specifically, the pattern information determination unit 48 performs pattern matching with the pattern information stored in the pattern information table of the pattern information table storage unit 71, and based on the safety evaluation result corresponding to the matched pattern information, It is judged whether or not the acquired pattern information is safe.
Specifically, the pattern information determination unit 48 determines whether the extracted pattern information matches the pattern information stored in the pattern information table of the pattern information table storage unit 71. Then, if there is matching pattern information in the pattern information stored in the pattern information table of the pattern information table storage unit 71, the pattern information determination unit 48 matches the pattern information table of the matched pattern information table storage unit 71. The safety evaluation result corresponding to the pattern information stored in the pattern information is referred to, and the safety evaluation result is output as the pattern information safety determination result.

The presentation information acquisition unit 49 converts the data into data based on the pattern information and acquires the presentation information.
The data converted based on the pattern information specifies (1) the location of the presentation information to be executed. (2) The content of the process executed by the presentation information is designated. (3) The program itself can be executed.
For example, in the case of (2) and (3), the presentation information is acquired from the storage unit 19, and in the case of (1), the presentation information is acquired from the outside via the communication unit 20.

The presentation information determination unit 50 determines the presentation information acquired by the presentation information acquisition unit 49. Specifically, the presentation information determination unit 50 performs presentation matching with the presentation information stored in the presentation information table, and the obtained presentation information is safe based on the safety evaluation result corresponding to the matched presentation information. Whether or not safety is determined.
Specifically, the presentation information determination unit 50 determines whether the acquired presentation information matches the presentation information stored in the presentation information table of the presentation information table storage unit 72. Then, when there is matching presentation information in the presentation information stored in the presentation information table of the presentation information table storage unit 72, the presentation information determination unit 50 presents the presentation information table of the matching presentation information table storage unit 72. With reference to the safety evaluation result corresponding to the presentation information stored in, the safety evaluation result is output as the safety judgment result of the presentation information.

  The presentation information content execution unit 51 is specified if, for example, the presentation information is an image display, the image is displayed on the output unit 18, or the presentation information is a transfer of data stored in the storage unit 19. The content corresponding to the presentation information that is transferred to the transfer destination and the data stored in the storage unit 19 is transferred, or the presentation information is a URL, the URL is accessed and displayed on the output unit 18 via a browser or the like. Execute the process.

[First Embodiment]
Next, a flow of pattern information confirmation processing executed by the imaging apparatus 1 in the present embodiment will be described with reference to FIG.
FIG. 5 is a flowchart for explaining the flow of pattern information confirmation processing executed by the imaging apparatus 1 of FIG. 3 having the functional configuration of FIG.

A user who operates the input unit 17 of the imaging apparatus 1 can perform a predetermined operation instructing the start of the pattern information confirmation process.
The pattern information confirmation process is started in response to a predetermined operation by such a user, and the following process is executed.

In step S1, the mode setting unit 43 determines whether or not the set mode is the extended mode.
When the set mode is not the extended mode, the shooting mode or other mode is determined as NO in step S1, and the process proceeds to A (step S31 in FIG. 6). The process A (the process after step S31 shown in FIG. 6) will be described later.
On the other hand, when the set mode is the extended mode, it is determined as YES in Step S1, and the process proceeds to Step S2.

  In step S2, the output unit 18 performs live view display, which is the initial setting of the augmented reality mode. Specifically, the output control unit 46 performs live view display, that is, live view display control that causes the output unit 18 to sequentially display and output captured image data acquired in real time from the imaging unit 16.

In step S3, the input detection unit 42 determines whether or not a half-press operation has been performed on the release button.
If the release button is not pressed halfway, it is determined as NO in step S3, and the process returns to step S3.
On the other hand, if the release button is pressed halfway, it is determined as YES in step S3, and the process proceeds to step S4.

  In step S4, the imaging control unit 44 performs AF processing. The imaging control unit 44 controls the imaging unit 16 to perform processing related to AF such as focusing.

In step S5, the pattern information extraction unit 47 determines whether a marker has been identified. That is, the pattern information extraction unit 47 searches for a marker that is pattern information from the acquired captured image. If there is a marker as a result of the search, the pattern information extraction unit 47 determines that the marker has been identified.
If the marker has not been identified, NO is determined in step S5, and the process returns to step S3.
On the other hand, when a marker is identified, it is judged as YES in Step S5 and processing progresses to Step S6.

  In step S6, the output unit 18 identifies and displays the marker. That is, the output control unit 46 controls the output unit 18 so that the marker identified in step S5 is displayed to the user so as to be visually identifiable. Specifically, the output unit 18 displays the identified marker so as to be surrounded by a frame (for example, display of “”).

In step S7, the input detection unit 42 determines whether or not the release button has been fully pressed.
If the release button has not been fully pressed, NO is determined in step S7, and the process returns to step S3.
On the other hand, if the release button is fully pressed, it is determined as YES in Step S7, and the process proceeds to Step S8.

In step S8, the pattern information extraction unit 47 determines whether there are a plurality of identified markers.
If there are not a plurality of markers, NO is determined in step S8, and the process proceeds to step S9.

  In step S9, the pattern information determination unit 48 performs a pattern matching check of the identified marker. Specifically, the pattern information determination unit 48 performs pattern matching between the pattern information of the identified marker and the pattern information stored in the pattern information table storage unit 71. A pattern matching check of the identified marker is performed, and the process proceeds to step S10.

In step S10, the pattern information determination unit 48 determines whether the safety of the marker has been confirmed. Specifically, the pattern information determination unit 48 determines whether or not the marker is determined to be a safe marker as a result of pattern matching.
If it is confirmed that the marker is not safe, that is, a dangerous marker, NO is determined in step S10, and the process proceeds to C (step S15). The processing after step S15 will be described later.
If it is confirmed that the marker is safe, YES is determined in step S10, and the process proceeds to step S17. The processing after step S17 will be described later.

  On the other hand, if there are a plurality of markers, YES is determined in step S8, and the process proceeds to step S11.

  In step S11, the pattern information determination unit 48 performs a pattern matching check of the identified marker. Specifically, the pattern information determination unit 48 performs pattern matching between the pattern information of the identified marker and the pattern information stored in the pattern information table storage unit 71. A pattern matching check of the identified marker is performed, and the process proceeds to step S12.

In step S12, the pattern information determination unit 48 determines whether or not all markers have been checked. That is, the pattern information determination unit 48 determines whether pattern matching has been performed for all the identified markers.
If all the markers have not been checked, it is determined as NO in step S12, and the process returns to step S11 to perform pattern matching for the unchecked markers.
On the other hand, if all the markers have been checked, YES is determined in step S12, and the process proceeds to step S13.

In step S13, the pattern information determination unit 48 determines whether there is a marker that cannot be checked. Specifically, the pattern information determination unit 48 determines whether there is pattern information that is not stored in the pattern information table storage unit 71 as a result of pattern matching, and there is a marker that cannot be checked.
If there is no uncheckable marker, it is determined as NO in step S13, and the process proceeds to step S10.
On the other hand, if there is a marker that cannot be checked, it is determined as YES in step S13, and the process proceeds to step S14.

In step S14, the pattern information determination unit 48 determines whether or not the safety of the checkable marker has been confirmed. Specifically, the pattern information determination unit 48 determines whether or not all the markers that have been checked for pattern matching are determined to be safe markers.
When the safety of the checkable marker is not confirmed, that is, when a dangerous marker that is not safe is confirmed in at least some of the markers, NO is determined in step S14, and the process proceeds to step S15.

  In step S15, the output unit 18 displays an alert. Specifically, the output control unit 46 displays an alert message that prompts the user to warn, such as “Dangerous marker is included” or “Marker safety could not be confirmed.” 18 is controlled. Thereafter, the process returns to step S3.

  On the other hand, when the safety of the checkable marker is confirmed, that is, when all the markers except the marker that cannot be checked are confirmed to be safe, it is determined as YES in Step S14, and the process proceeds to Step S16. Proceed to

In step S16, the main control unit 41 determines whether or not the user has approved the safety confirmation of the uncheckable marker. That is, from the result that the input detection unit 42 has detected the input from the input unit 17 after the main control unit 41 performs a display for prompting the user to confirm the safety confirmation of the uncheckable marker, It is determined whether or not the user has approved the safety confirmation of the uncheckable marker.
If the user does not approve the safety confirmation of the uncheckable marker, NO is determined in step S16, the process proceeds to step S15, and the output unit 18 displays, for example, “The safety of the marker could not be confirmed”. An alert is displayed to the effect that the safety confirmation of the uncheckable marker has not been accepted.
On the other hand, if the user approves the safety check of the uncheckable marker, YES is determined in step S16, and the process proceeds to step S17.

  In step S <b> 17, the presentation information acquisition unit 49 acquires marker presentation information. Specifically, the presentation information acquisition unit 49 acquires marker presentation information from a server or the like connected to the network via the communication unit 20.

  In step S <b> 18, the presentation information determination unit 50 checks the safety of the acquired presentation information. Specifically, the presentation information determination unit 50 performs pattern matching between the presentation information stored in the presentation information table storage unit 72 and the acquired presentation information.

In step S19, the presentation information determination unit 50 determines whether or not the safety of the acquired presentation information has been confirmed.
If safety is not confirmed, NO is determined in step S19, and the process proceeds to C (proceeds to step S15).
On the other hand, if safety is confirmed, YES is determined in step S19, and the process proceeds to step S20.

  In step S20, the presentation information content execution unit 51 executes the content of the acquired presentation information. Specifically, when the content of the acquired presentation information is an image based on the pattern information, the presentation information content execution unit 51 acquires an image as shown in FIG. Display and output an image. Thereafter, the pattern information confirmation process ends.

Next, a flow of processing when a mode other than the extended mode is set in the pattern information confirmation processing executed by the imaging apparatus 1 according to the present embodiment will be described with reference to FIG.
FIG. 6 is a flowchart for explaining the detailed flow of the process when the mode other than the augmented reality mode is set in the pattern information confirmation process of FIG.

In step S1 of FIG. 5, when it is determined that the mode is not the augmented reality mode and the process proceeds to step A, step S31 of FIG.
In step S31, the mode setting unit 43 determines whether the set mode is the shooting mode.
If the set mode is not the shooting mode, NO is determined in step S31, and the process proceeds to step S32.

In step S32, the mode setting unit 43 determines whether or not there is another mode such as a playback mode for sending a playback image or performing a slide show.
If the set mode is not another mode, NO is determined in step S32, and the process proceeds to B (returns to step S1 in FIG. 5).
On the other hand, if the set mode is any other mode, YES is determined in step S32, and the process proceeds to step S33.

  In step S33, the main control unit 41 performs processing according to a mode such as image forwarding and slide show display in the playback mode, and then the process proceeds to B (returns to step S1 in FIG. 5).

If the set mode is the shooting mode, YES is determined in step S31, and step S31 in FIG. 6 is started.
In step S34, the output unit 18 performs live view display, which is the initial setting of the shooting mode. Specifically, the output control unit 46 performs live view display, that is, live view display control in which the output unit 18 sequentially displays and outputs captured image data acquired in real time from the imaging unit 16.

In step S35, the input detection unit 42 determines whether or not a half-press operation has been performed on the release button.
If the release button is not pressed halfway, it is determined as NO in Step S35, and the process proceeds to Step S40. The process of step S40 will be described later.
On the other hand, if the release button has been pressed halfway, YES is determined in step S35, and the process proceeds to step S36.

  In step S36, the imaging control unit 44 performs AF, AE, and AWB processing. The imaging control unit 44 controls the imaging unit 16 to perform processing related to AF, AE, and AWB.

In step S37, the input detection unit 42 determines whether or not the release button has been fully pressed.
If the release button has not been fully pressed, NO is determined in step S37, and the process returns to step S35.
On the other hand, if the release button is fully pressed, it is determined as YES in Step S37, and the process proceeds to Step S38.

  In step S38, the imaging control unit 44 performs imaging processing. Specifically, the imaging control unit 44 controls the imaging unit 16 to perform imaging processing.

  In step S39, the captured image acquisition unit 45 stores the captured image. Specifically, the captured image acquisition unit 45 stores the captured image acquired from the imaging unit 16 in the removable medium 31 via the storage unit 19 or the drive 21. Thereafter, the process ends.

If it is determined in step S35 that the release button has not been pressed halfway, the main control unit 41 determines in step S40 whether or not a mode change operation has been performed. Specifically, the main control unit 41 determines whether or not the operation on the input unit 17 detected by the input detection unit 42 is an operation related to mode change.
If there is a mode change operation, YES is determined in step S40, the process proceeds to B, and the process returns to step S1 in FIG.
On the other hand, if it is not a mode change operation, NO is determined in step S35, and the process proceeds to step S41.

In step S <b> 41, the main control unit 41 determines whether it is another operation such as an exposure setting operation or a zoom setting operation.
When it is not other operation, it is judged as NO in Step S41, and processing is ended after that.
On the other hand, if it is another process, YES is determined in step S41, and the process proceeds to step S42.

  In step S42, the main control unit 41 performs processing according to operations such as exposure setting in the exposure setting operation and zoom processing in the zoom setting operation, and then the processing ends.

  Therefore, the imaging apparatus 1 can safely use the augmented reality function.

[Second Embodiment]
Next, the flow of the light emitter information confirmation process executed by the imaging apparatus 1 in the present embodiment will be described with reference to FIG.
FIG. 7 is a flowchart for explaining the flow of the illuminant information confirmation process executed by the imaging apparatus 1 according to the present embodiment.

Here, the light emitter information confirmation process is a series of processes for confirming information from a light emitter recognized in visible light communication and executing a process according to the information from the light emitter.
In this embodiment, the light emitter information confirmation process is executed when the mode of the imaging device 1 is the visible light communication mode. In this case, the storage unit 19 stores information on whether or not the visible light communication is safe, and the safety is determined by comparing the information with the visible light communication actually performed. It will be.

The illuminant information confirmation process is different from the pattern information confirmation process of the first embodiment, as shown in FIG. 7, in that the trigger for exerting the augmented reality function is a marker or a illuminant. Processes other than the process of recognizing the blinking pattern of the light emitter are basically the same as the pattern information confirmation process of the first embodiment. That is, the processes in steps S51 to S70 in FIG. 7 correspond to the processes in steps S1 to S20 in FIG. 5, and D in FIG. 7 corresponds to C in FIG.
Specifically, in step S51, it is determined whether or not the visible light communication mode is used instead of the augmented reality mode in step S1.
In step S55, a light emitter for visible light communication is identified instead of the marker in step S5.
In step S56, the light emitter is identified and displayed instead of the marker in step S6. That is, the output unit 18 displays the identified light emitter. At this time, the output unit 18 displays the identified light emitter so as to be surrounded by a frame (for example, a display of “”).
In step S58, it is determined whether or not there are a plurality of light emitters instead of the marker in step S8.
In step S59 and step S61, pattern matching of identification light emitters is performed in place of the identification markers in step S9 and step S11.
In step S62, it is determined whether or not all the light emitters are checked instead of all the markers in step S12.
In step S63, it is determined whether or not the presence or absence of an uncheckable light emitter has been confirmed in place of the uncheckable marker in step S13.
In step S64, it is determined whether or not the safety confirmation of the light emitter that can be checked is accepted in place of the checkable marker in step S14.
Further, regarding the alert display in step S65, an alert display such as “Dangerous illuminant is included” or “Safety of illuminant could not be confirmed” is displayed.

  Accordingly, the imaging device 1 is configured to be capable of visible light communication, and even when using the visible light communication illuminant as a signal to exhibit the augmented reality function, the augmented reality function can be safely used. be able to.

[Modification]
Next, a modification example regarding determination of pattern information (light emitter information) and presentation information in the imaging device 1 of the first and second embodiments will be described with reference to FIG.
FIG. 8 is a system configuration diagram illustrating a system that acquires each information table in the imaging apparatus 1. In addition, the code | symbol about the structure same as the above-mentioned embodiment is the same, and description is abbreviate | omitted.

  The example of FIG. 8 includes a mobile terminal 100 having an imaging function, a communication function via a network, a base station 101, a network N such as the Internet, and a server 102.

The portable terminal 100 is configured to communicate with the base station 101 and connect to a network, and to exchange data with the server 102.
The server 102 is connected to the network N, and a pattern information table storage unit (a storage unit storing a light emitter information table) and a storage unit corresponding to a presentation information table storage unit are provided in one area of the storage unit. Further, the pattern information table and the presentation information table stored in the server 102 are updated to the latest information as needed by information provided from the network.

In the system configured as described above, for example, when the augmented reality function is exhibited by the mobile terminal 100, each table is provided from the server 102 so that pattern information and presentation information can be determined. Can be configured. Further, the determination itself of the pattern information and the presentation information can be performed by the server 102, and only the determination result can be transmitted to the mobile terminal 100.
By configuring the system in this way, the portable terminal 100 can reduce processing load and storage capacity, and can always make determinations with new information.

As described above, the imaging device 1 of the present embodiment includes the captured image acquisition unit 45, the pattern information extraction unit 47, the pattern information determination unit 48, and the presentation information acquisition unit 49, and further captures an image. Imaging means.
The captured image acquisition unit 45 acquires captured image data including a subject (marker) that presents information represented by a pattern.
The pattern information extraction unit 47 identifies the subject from the image acquired by the captured image acquisition unit 45.
The pattern information determination unit 48 checks whether the pattern representing the subject identified by the pattern information extraction unit 47 is safe.
The presentation information acquisition unit 49 acquires information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the pattern information determination unit 48.
The captured image acquisition unit 45 acquires the captured image captured by the imaging unit 16.
In this case, the image capturing apparatus 1 captures information presented by the subject simply by inspecting the pattern of the imaged subject in the inspection apparatus that captures the information represented by the pattern and presents the information and acquires the information. Safety can be confirmed before.

In addition, the pattern information determination unit 48 compares the pattern expressing the subject with the pattern information that has been evaluated for safety in advance, to determine whether or not the pattern expressing the subject is safe. Perform the inspection.
In addition, the pattern information determination unit 48 performs inspection by comparing the pattern information with the first table associated with the safety evaluation corresponding to the pattern information.
In this case, since the imaging apparatus 1 performs inspection by collating with the pattern information table, it can be easily inspected, and the latest information can always be inspected by updating the table.

The imaging apparatus 1 further includes a presentation information determination unit 50 and a presentation information content execution unit 51.
The presentation information determination unit 50 checks whether the information presented by the subject acquired by the presentation information acquisition unit 49 is safe.
The presentation information content execution unit 51 executes processing included in the information presented by the subject when it is determined by the examination performed by the presentation information determination unit 50 that the information presented by the subject is safe.
In this case, since the imaging apparatus 1 further inspects the presentation information acquired by checking the safety by collating with the pattern information table, the imaging apparatus 1 can perform more reliable safety confirmation.

The presentation information determination unit 50 checks whether the information presented by the subject is safe by comparing the information presented by the subject with the presentation information whose safety has been evaluated in advance. .
In addition, the presentation information determination unit 50 performs inspection by comparing the presentation information with a second table in which safety evaluation corresponding to the presentation information is associated.
In this case, since the imaging apparatus 1 performs the inspection by collating with the presentation information table, the inspection can be easily performed, and the latest information can always be inspected by updating the table.

In the imaging apparatus 1, when a plurality of subjects are identified by the pattern information extraction unit 47, and it is impossible to determine whether or not some of the plurality of subjects are safe by the pattern information determination unit 48. In addition, in accordance with the determination result of other subjects for which it can be determined whether or not it is safer, an inspection is performed as to whether or not the subject that could not be determined is safe.
In this case, even when a plurality of subjects are identified and the safety of some of them cannot be determined, the imaging apparatus 1 can estimate the safety according to the results of other determined subjects.

The imaging apparatus 1 determines that the other subject is safe by the inspection performed by the pattern information determination unit 48, and determines that the subject that could not be determined according to the result is safe. The output unit 18 confirms whether or not the information presented by the subject that cannot be determined by the acquisition unit is acquired.
In this case, the imaging device 1 allows the user to confirm the estimated safety and determine acquisition of the presentation information.

The imaging apparatus 1 further includes a communication unit 20 that performs communication with an external apparatus (server 102) that performs inspection. The imaging apparatus 1 performs processing related to the inspection unit (the pattern information determination unit 48 and the presentation information determination unit 50) with an external device via the communication unit 20.
In this case, since the imaging apparatus 1 performs inspection with an external apparatus such as the server 102 that performs safety inspection via communication, it is possible to reduce the load on the equipment that performs the processing.

The subject is a marker for realizing augmented reality.
In this case, the imaging apparatus 1 can easily perform a safety inspection in augmented reality.

The subject is a light emitter for visible light communication that performs communication by a blinking pattern of visible light.
In this case, the imaging apparatus 1 can easily perform a safety inspection in visible light communication.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

  In the above-described embodiment, the marker is displayed by the two-dimensional code, but the present invention is not limited to this. The marker may be expressed by various types of code information such as a one-dimensional code such as a barcode or a two-dimensional code such as a QR code (registered trademark) as long as the information can be acquired. Furthermore, the marker does not have to be a code, and may be a combination of feature points (positional relationship of feature points, a combination of feature description vectors). If a combination of feature points is used, a subject that can be pattern-matched such as a signboard, a logo, a post, and a face can be used as a marker. In other words, the subject can be configured with various codes, and in this case, the imaging apparatus 1 can easily perform a safety inspection in reading the code.

  In the above-described embodiment, the identification of the marker and the identification of the illuminant for visible light communication are started by half-pressing the release button, but the live when it is determined that the augmented reality mode or the visible light communication mode is selected. During the view display, the safety may be confirmed by always identifying the marker or the light emitter for visible light communication.

  In the above-described embodiment, safety confirmation is determined using pattern matching using a data table, but the present invention is not limited to this. For example, the pattern information and the presentation information to be checked for safety may be regarded as virus determination, and may be determined using a known virus determination method.

  In the above-described embodiment, the presentation information is an image, a URL, and a program. In this case, for example, it is possible to determine harmful images, malicious sites, and virus contamination that you do not want to show to your child.

  In addition, when some information of a plurality of pattern information and presentation information cannot be determined because there is no matching target in the table data, it may be output as a comprehensive determination result using the determination result of other information. Good. For example, if some of the determination results are determined to be dangerous, it may be determined as dangerous as a whole.

In the above-described embodiment, the imaging apparatus 1 to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having a pattern information confirmation process and a light emitter information confirmation process function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 4 is merely an example, and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional block is used to realize this function is not particularly limited to the example of FIG.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only configured by the removable medium 31 of FIG. 3 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in this. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preinstalled in the apparatus main body includes, for example, the ROM 12 in FIG. 3 in which the program is recorded, the hard disk included in the storage unit 19 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall device configured by a plurality of devices, a plurality of means, and the like.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalent scope thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
Image acquisition means for acquiring data of an image including a subject represented by a pattern and presenting information;
Identifying means for identifying the subject from the image obtained by the image obtaining means;
First inspection means for inspecting whether or not a pattern representing the subject identified by the identification means is safe;
Information acquisition means for acquiring information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the first inspection means;
An information processing apparatus comprising:
[Appendix 2]
Whether the pattern representing the subject is safe by comparing the pattern representing the subject with pattern information whose safety has been evaluated in advance. The information processing apparatus according to appendix 1, wherein an inspection is performed.
[Appendix 3]
The first inspection unit performs the inspection by comparing the pattern information with a first table in which safety evaluation corresponding to the pattern information is associated with the pattern information. Information processing device.
[Appendix 4]
Second inspection means for inspecting whether or not the information presented by the subject acquired by the information acquisition means is safe;
Execution means for executing processing included in the information presented by the subject when the information presented by the subject is determined to be safe by the examination performed by the second examination means;
The information processing apparatus according to any one of appendices 1 to 3, further comprising:
[Appendix 5]
The second inspection means performs an inspection as to whether or not the information presented by the subject is safe by comparing the information presented by the subject with the presentation information that has been evaluated for safety in advance. The information processing apparatus according to appendix 4, which is performed.
[Appendix 6]
The supplementary note 5 is characterized in that the second inspection means performs an inspection by collating the presented information with a second table associated with a safety evaluation corresponding to the presented information. Information processing device.
[Appendix 7]
When the plurality of subjects are identified by the identification unit, and when it is impossible to determine whether or not some of the plurality of subjects are safe by the first inspection unit, the first Third inspection means for inspecting whether or not the subject that could not be discriminated is safe according to the discrimination result of another subject that can be discriminated whether or not it is safe by one inspection means The information processing apparatus according to any one of supplementary notes 1 to 6, further comprising:
[Appendix 8]
The information acquisition is performed when it is determined that the other subject is safe by the inspection performed by the third inspection unit, and the unidentifiable subject is determined to be safe according to the result. The information processing apparatus according to appendix 7, further comprising confirmation means for confirming to a user whether or not information to be presented by the subject that cannot be determined is acquired by the means.
[Appendix 9]
An image pickup means for picking up an image;
The information processing apparatus according to any one of appendices 1 to 8, wherein the image acquisition unit acquires a captured image captured by the imaging unit.
[Appendix 10]
It further comprises a communication means for communicating with an external device that performs inspection,
The information processing apparatus according to any one of appendices 1 to 9, wherein the processing related to the inspection unit is performed by the external device via the communication unit.
[Appendix 11]
The information processing apparatus according to any one of appendices 1 to 10, wherein the subject is a marker for realizing augmented reality.
[Appendix 12]
The information processing apparatus according to any one of appendices 1 to 10, wherein the subject is a light emitter for visible light communication that performs communication by a blinking pattern of visible light.
[Appendix 13]
The information processing apparatus according to any one of appendices 1 to 10, wherein the subject is code information.
[Appendix 14]
An information processing method executed by the information processing apparatus,
An image acquisition step of acquiring data of an image including a subject which is expressed by a pattern and presents information;
An identification step of identifying the subject from the image acquired by the image acquisition step;
A first inspection step for inspecting whether or not the pattern representing the subject identified by the identification step is safe;
An information acquisition step of acquiring information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the first inspection step;
An information processing method comprising:
[Appendix 15]
Computer
An image acquisition function for acquiring image data including a subject that presents information while being represented by a pattern,
An identification function for identifying the subject from the image acquired by the image acquisition function;
A first inspection function for inspecting whether or not a pattern representing the subject identified by the identification function is safe;
An information acquisition function for acquiring information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the first inspection function;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... Input-output interface, 16 ... Imaging part, 17 ... Input unit, 18 ... output unit, 19 ... storage unit, 20 ... communication unit, 21 ... drive, 31 ... removable media, 41 ... main control unit, 42 ... input Detection unit, 43 ... Mode setting unit, 44 ... Imaging control unit, 45 ... Captured image acquisition unit, 46 ... Output control unit, 47 ... Pattern information extraction unit, 48 ... Pattern Information determination unit 49... Presentation information acquisition unit 50. Presentation information determination unit 51. Presentation information content execution unit 71. Pattern information table storage unit 72. Presentation information table storage Part, 100... Portable terminal, 101... Base station, 102. Server, N ··· network

Claims (15)

Image acquisition means for acquiring data of an image including a subject represented by a pattern and presenting information;
Identifying means for identifying the subject from the image obtained by the image obtaining means;
First inspection means for inspecting whether or not a pattern representing the subject identified by the identification means is safe;
Information acquisition means for acquiring information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the first inspection means;
An information processing apparatus comprising:   Whether the pattern representing the subject is safe by comparing the pattern representing the subject with pattern information whose safety has been evaluated in advance. The information processing apparatus according to claim 1, wherein an inspection is performed.   3. The inspection according to claim 2, wherein the first inspection unit performs inspection by collating the pattern information with a first table associated with safety evaluation corresponding to the pattern information. The information processing apparatus described. Second inspection means for inspecting whether or not the information presented by the subject acquired by the information acquisition means is safe;
Execution means for executing processing included in the information presented by the subject when the information presented by the subject is determined to be safe by the examination performed by the second examination means;
The information processing apparatus according to claim 1, further comprising:   The second inspection means performs an inspection as to whether or not the information presented by the subject is safe by comparing the information presented by the subject with the presentation information that has been evaluated for safety in advance. The information processing apparatus according to claim 4, wherein the information processing apparatus performs the processing.   The said 2nd test | inspection means performs a test | inspection by collating the said presentation information and the 2nd table with which safety evaluation corresponding to this presentation information was matched. The information processing apparatus described.   When the plurality of subjects are identified by the identification unit, and when it is impossible to determine whether or not some of the plurality of subjects are safe by the first inspection unit, the first Third inspection means for inspecting whether or not the subject that could not be discriminated is safe according to the discrimination result of another subject that can be discriminated whether or not it is safe by one inspection means The information processing apparatus according to claim 1, further comprising: an information processing apparatus according to claim 1.   The information acquisition is performed when it is determined that the other subject is safe by the inspection performed by the third inspection unit, and the unidentifiable subject is determined to be safe according to the result. The information processing apparatus according to claim 7, further comprising confirmation means for confirming to a user whether or not information to be presented by the subject that could not be identified is acquired by the means. An imaging means for capturing an image;
The information processing apparatus according to claim 1, wherein the image acquisition unit acquires a captured image captured by the imaging unit. It further comprises a communication means for communicating with an external device that performs inspection,
The information processing apparatus according to claim 1, wherein the processing related to the inspection unit is performed by the external device via the communication unit.   The information processing apparatus according to claim 1, wherein the subject is a marker for realizing augmented reality.   The information processing apparatus according to claim 1, wherein the subject is a visible light communication light emitter that performs communication by a blinking pattern of visible light.   The information processing apparatus according to claim 1, wherein the subject is code information. An information processing method executed by the information processing apparatus,
An image acquisition step of acquiring data of an image including a subject which is expressed by a pattern and presents information;
An identification step of identifying the subject from the image acquired by the image acquisition step;
A first inspection step for inspecting whether or not the pattern representing the subject identified by the identification step is safe;
An information acquisition step of acquiring information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the first inspection step;
An information processing method comprising: Computer
An image acquisition function for acquiring image data including a subject that presents information while being represented by a pattern,
An identification function for identifying the subject from the image acquired by the image acquisition function;
A first inspection function for inspecting whether or not a pattern representing the subject identified by the identification function is safe;
An information acquisition function for acquiring information presented by the subject when the pattern representing the subject is determined to be safe by the inspection performed by the first inspection function;
A program characterized by functioning as

Images