JP2011009804A - Photographing device, portable terminal, transmitter, photographing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem that a conventional photographing device cannot acquire information about a subject upon photographing.SOLUTION: A photographing device includes a first photographing part 100 for photographing a subject to acquire an image, a second photographing part 101 for photographing one or more light sources including an optical radio communication light source to acquire an image, a light source position detecting part 102 for detecting the optical radio communication light source in a second photographed image acquired by the second photographing part 101 and acquiring the position information of the optical radio communication light source, an information acquiring part 103 for acquiring communication light incident position information of the position where a light output from the optical radio communication light source is incident from the position information of the optical radio communication light source acquired by the light source position detecting part 102, and acquiring target transmission information to be transmitted by optical radio transmission from the optical radio communication light source at the incident position shown by the communication light incident position information, and an output part 104 for outputting the first photographed image acquired by the first photographing part 100 and the information acquired by the information acquiring part 103.

Description

  The present invention relates to a photographing apparatus for photographing a subject.

As a conventional photographing apparatus, there has been known a digital camera that includes an image pickup unit that picks up a subject image and records image data picked up by the image pickup unit (see, for example, Patent Document 1).
Further, as a conventional photographing apparatus, a digital camera including a photographing date acquisition unit that acquires a photographing date of an arbitrary image has been known (for example, see Patent Document 1).

Japanese Patent Laying-Open No. 2005-354606 (first page, FIG. 1 etc.) Japanese Patent Laying-Open No. 2005-303453 (first page, FIG. 1 etc.)

  However, in a conventional photographing apparatus, only information on an image obtained by photographing the appearance of a subject to be photographed, information on the date and time of photographing, equipment used for photographing, aperture, exposure, and the like is acquired. However, there is a problem that it is impossible to acquire detailed explanation about the subject, for example, information such as what the subject is.

  The image capturing apparatus of the present invention captures an image by capturing at least one light source including a first image capturing unit that captures an image by capturing a subject and an optical wireless communication light source that is a light source that performs optical wireless communication. An imaging unit, and a light source position detection unit that detects an optical wireless communication light source in a second captured image that is an image acquired by the second imaging unit, and acquires positional information that is position information of the optical wireless communication light source; From the position information of the optical wireless communication light source acquired by the light source position detection unit, communication light incident position information that is information on a position where light output from the optical wireless communication light source is incident is acquired, and the communication light incident position is acquired. An information acquisition unit that acquires transmission target information that is information transmitted by optical wireless communication from the optical wireless communication light source at an incident position indicated by information; a first captured image that is an image acquired by the first imaging unit; Acquired by the information acquisition unit Is an imaging apparatus having an output unit for outputting the transmission target information.

  With this configuration, it is possible to acquire an image obtained by photographing a subject and information on the subject.

  Further, in the imaging apparatus of the present invention, in the imaging apparatus, the information acquisition unit acquires information transmitted by optical wireless communication in synchronization with imaging performed by the first imaging unit, and the output unit includes: It is an imaging device that associates and outputs a first captured image that is an image acquired by the first imaging unit and information acquired by the information acquisition unit in synchronization with the imaging of the first captured image.

  With this configuration, it is possible to acquire information on the subject transmitted at the time of photographing together with the image obtained by photographing the subject in a timely manner.

  In the photographing apparatus of the present invention, in the photographing apparatus, the information acquisition unit recognizes a start point and an end point of optical wireless communication information transmitted by optical wireless communication at a time when photographing is performed by the first photographing unit. Then, this is an imaging apparatus that acquires optical wireless communication information in a range from the recognized start point to the end point as information transmitted in synchronization with imaging.

  With this configuration, it is possible to acquire information on the subject transmitted at the time of photographing together with the image obtained by photographing the subject in a timely manner.

  Moreover, the imaging device of the present invention is the imaging device, wherein the transmission target information is information relating to a subject to be imaged by the first imaging unit.

  With this configuration, it is possible to acquire an image obtained by photographing a subject and information on the subject.

  In addition, the imaging device of the present invention is a transmission target information acquisition unit that acquires transmission target information that is information received by the imaging device and is information to be transmitted by optical wireless communication. A modulation unit that modulates transmission target information acquired by the target information acquisition unit, and information that is used to detect the position of a light source that performs optical wireless communication, and a marker having a frequency different from the frequency at which the modulation unit modulates A marker signal output unit for outputting a signal, a transmission target information output from the modulation unit, a superimposing unit for superimposing a marker signal output from the marker signal output unit, and a transmission target information and a marker superimposed by the superimposing unit And a transmission unit that transmits information including a signal.

  With this configuration, it is possible to determine whether the light source that performs optical wireless communication using transmitted information is a light source that performs optical communication by detecting a marker signal having a frequency different from that of the transmission target information. It is possible to easily determine whether or not the imaging device is a light source that performs optical wireless communication.

  According to the photographing apparatus or the like according to the present invention, it is possible to acquire an image obtained by photographing a subject and information on the subject.

The figure for demonstrating the structure of the imaging device in this Embodiment Flow chart for explaining the operation of the photographing apparatus Flow chart for explaining the operation of the photographing apparatus The figure for demonstrating the marker signal of the imaging device Front perspective view of a portable terminal equipped with the same photographing device Rear perspective view of a portable terminal equipped with the same photographing device Schematic showing an example of use of a portable terminal equipped with the same photographing device The figure which shows the 2nd picked-up image for demonstrating operation | movement of the imaging device. The figure which shows the example which binarized the 2nd picked-up image for demonstrating operation | movement of the imaging device The figure which shows an example of the light source coordinate management table | surface of the imaging device The figure which shows an example of the visible light communication light source management table | surface of the imaging device The figure which shows the permeation | transmission area | region of a liquid crystal panel for demonstrating operation | movement of the imaging device The figure which shows the state in which the light which permeate | transmitted the liquid crystal panel is received for demonstrating operation | movement of the imaging device The figure which shows an example of the 1st picked-up image image | photographed with the imaging device. The figure which shows an example of the exhibit related information which the imaging device acquired The figure which shows the example of a display of the imaging device Block diagram of the transmitter Schematic diagram showing an example of the external appearance of the computer system The figure which shows an example of a structure of the computer system

  Hereinafter, embodiments of a photographing apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment 1)
FIG. 1 is a block diagram of a photographing apparatus according to the present embodiment.

  The imaging apparatus 1 according to the present embodiment is an apparatus that captures a subject and receives information about the subject that is transmitted from a light source that performs visible light communication installed around the subject.

  Visible light communication is an apparatus that transmits data by changing the intensity of light emitted from a light source in accordance with a signal obtained by modulating data to be transmitted. In other words, it is a technique for transmitting information by fluctuations in light intensity. Note that this variation in light intensity is a concept including blinking of light. However, the blinking of light here may be considered as a change in light intensity. Here, a light source that performs visible light communication is referred to as a visible light communication light source. In the present embodiment, the visible light communication light source is preferably an LED light source. In addition, in order to prevent a false detection in which a light source other than a visible light communication light source is erroneously detected as a visible light communication light source, a light source other than a visible light communication light source is also a light source with little flickering, for example, other than a fluorescent lamp. An LED light source or an incandescent lamp is preferred. Since the visible light communication technique such as a configuration for transmitting information via the visible light communication light source is a known technique, a detailed description thereof will be omitted.

  The imaging apparatus 1 includes a first imaging unit 100, a second imaging unit 101, a light source position detection unit 102, an information acquisition unit 103, an output unit 104, a first lens 200, a second lens 201, and a beam splitter 202.

  The first imaging unit 100 includes a first imaging element 1001 and first image processing means 1002.

  The second imaging unit 101 includes a second image sensor 1011 and second image processing means 1012.

  The light source position detection unit 102 includes a light source detection unit 1021 and a light source selection unit 1022.

  The information acquisition unit 103 includes incident position information acquisition means 1031, selective transmission means 1032, light receiving means 1033, information acquisition means 1034, and a third lens 203.

  The selective transmission means 1032 includes a liquid crystal panel 10321 and a light shielding control means 10322.

  The first lens 200 is a condensing unit that condenses the light emitted from the subject and the light reflected by the subject and makes the light incident on the first photographing unit 100 in the photographing apparatus 1. Specifically, the subject here is a subject to be captured by a user or the like to acquire an image by photographing. Here, the light condensed by the first lens 200 is incident on the first imaging unit 100.

  The first photographing unit 100 photographs a subject and acquires an image. Specifically, the first photographing unit 100 photographs a subject via the first lens 200 and acquires information on an image obtained by photographing the subject. In this embodiment, the image acquired by the first imaging unit 100 is referred to as a first captured image. The first captured image acquired by the first imaging unit 100 may be a color image, a gray scale image, or the like. Further, the first photographing unit 100 may photograph a still image or an image that is a moving image composed of a plurality of frames. The moving image described in this embodiment is a concept including a plurality of still images taken continuously along a time series. In this embodiment, in particular, the configuration of the first imaging unit 100 will be described by taking as an example the case where the first imaging unit 100 includes the first imaging element 1001 and the first image processing means 1002. The first imaging unit 100 can usually be realized by an imaging device, a GPU (Graphics Processing Unit), an MPU, a memory, or the like. The processing procedure of the first photographing unit 100 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The first image sensor 1001 outputs an electrical signal corresponding to the incident light. The first imaging element 1001 is an element configured by arranging a plurality of light receiving elements (not shown). Specifically, the first image sensor 1001 is a so-called image sensor. The first image sensor 1001 is, for example, a CCD or a CMOS. The first image sensor 1001 converts incident light into an electric signal corresponding to the intensity of the light, and outputs an electric signal whose waveform changes according to a change in the intensity of the incident light. Specifically, each light receiving element constituting the first image sensor 1001 outputs an electrical signal corresponding to the intensity of the incident light, and as a result, the first image sensor 1001 receives light from the image sensor. An electric signal corresponding to the intensity distribution of incident light on the surface is output. In general, the first image sensor 1001 can extract only an electric signal obtained by a light receiving element in a desired region according to the received light. In the present embodiment, light emitted from the subject or reflected from the subject is collected by the first lens 200 and is incident on the first image sensor 1001. A color filter such as RGB is usually provided on the incident surface of the first image sensor 1001. By selecting the color of light incident on each light receiving element with the color filter, the final image is finally obtained. A color image can be constructed. However, the color filter may be omitted as needed in the structure of the first image sensor 1001 and the light source detection. The color filter may be changed to a filter that transmits only light of a specific wavelength.

  The first image processing unit 1002 receives an electrical signal output from the first image sensor 1001 and forms an image of the subject. Specifically, the electrical signal output from the first image sensor 1001 is demodulated and converted into a digital signal or the like to constitute information on the image of the subject. Since the process in which the first image processing unit 1002 configures the information of the first captured image is a known technique, a detailed description thereof is omitted. The image formed by the first image processing unit 1002 is, for example, a moving image including one or more still images or images of a plurality of frames. The file format of the image which the 1st image processing means 1002 comprises is not ask | required. The first image processing unit 1002 temporarily stores the configured image information in a storage medium such as a memory (not shown). The first image processing unit 1002 can be realized by, for example, a DSP (Digital Signal Processor), a GPU, a memory, or the like. The processing procedure of the first image processing means 1002 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second lens 201 is a condensing unit that condenses light emitted from one or more light sources including a light source used for visible light communication and causes the light to enter the photographing apparatus 1. Specifically, the light condensed by the second lens 201 enters the second imaging unit 101. Instead of the second lens 201, other condensing means such as a concave mirror may be provided. Note that the second lens 201 has the optical axis of the first lens 200 directed toward the subject to be photographed, and the optical axis is the direction in which the light source is arranged, for example, the ceiling direction in which the light source is usually arranged. It is preferable to install it in the photographing apparatus 1 so as to face.

  The beam splitter 202 transmits the light incident through the second lens 201 to be incident on the second imaging unit 101, and further reflects the incident light to be incident on the information acquisition unit 103 described later. Specifically, the beam splitter 202 causes the transmitted light to enter the second image sensor 1011. The beam splitter 202 inputs the reflected light to the selective transmission means 1032 of the information acquisition unit 103 described later, more specifically, the liquid crystal panel 10321. Here, the half mirror is also considered as one of the beam splitters 202. In the present embodiment, a structure that allows light incident through the second lens 201 to be incident on both the second imaging unit 101 and the information acquisition unit 103 simultaneously or by switching. If provided, a configuration other than the beam splitter 202 may be provided. For example, light that is incident through the second lens 201 is provided by providing a flip-up type mirror used for a so-called single-lens reflex camera or the like instead of the beam splitter 202 and controlling the timing of the mirror's flip-up. May be switched and made incident on both the second imaging unit 101 and the information acquisition unit 103. In addition, the second imaging unit 101, the information acquisition unit 103, and the second lens 201 are moved relative to each other so that the second imaging unit 101, the information acquisition unit 103, It may be possible to switch and arrange them.

  The second imaging unit 101 captures one or more light sources including a visible light communication light source that is a light source that performs visible light communication, and acquires an image. Here, the acquired image captured by the second imaging unit 101 is referred to as a second captured image. The second photographing unit 101 photographs one or more light sources including a visible light communication light source via the second lens 201, and acquires information of an image obtained by photographing the one or more light sources. In addition, light sources other than a visible light communication light source are light sources, such as normal illumination in which information is not transmitted using light. The second captured image may be a color image as long as it can detect the position of a light source that performs visible light communication in the light source position detection unit 102, which will be described later. It may be a scale image or an image representing only the brightness of the image. The second photographing unit 101 may photograph a still image or a photographed image that is a moving image composed of a plurality of frames. In this embodiment, in particular, the configuration of the second imaging unit 101 will be described by taking as an example a case where the second imaging unit 101 includes the second imaging element 1011 and the second image processing unit 1012. The second imaging unit 101 can usually be realized by an imaging device, a GPU (Graphics Processing Unit), an MPU, a memory, or the like. The processing procedure of the second imaging unit 101 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The second image sensor 1011 outputs an electrical signal corresponding to the incident light. Since the configuration, processing, and the like of the second image sensor 1011 are the same as those of the first image sensor 1001 described above, detailed description thereof is omitted here. In the present embodiment, the light emitted from the visible light communication light source is collected by the second lens 201 or the like and is incident on the second image sensor 1011.

  The second image processing unit 1012 receives the electrical signal output from the second image sensor 1011 and configures the second captured image. Since the configuration, processing, and the like of the second image processing unit 1012 are the same as the configuration of the first image processing unit 1002 described above, detailed description thereof is omitted here.

  The light source position detection unit 102 detects a visible light communication light source in a second captured image that is an image acquired by the second imaging unit 101. And the positional information which is the information of the position of the detected visible light communication light source is acquired. The position of the visible light communication light source described here is the position of all or some of the pixels included in the image of the visible light communication light source in the second captured image, or the image of the visible light communication light source in the second captured image. These are the position of the center point of the existing area, the position of the outline of the area, the position representing the area such as each corner, and the like. The position information acquired by the light source position detection unit 102 is, for example, information on coordinates in the second captured image indicating the position of the detected visible light communication light source. The unit of the coordinate information is, for example, a pixel unit or a relative value with respect to the vertical or horizontal length of the captured image. Also, information indicating the position of a light receiving element (not shown) in the second image sensor 1011 that is an output source of the electrical signal used for obtaining the pixel of the visible light communication light source in the second photographed image, for example, coordinates It may be information.

  For example, the light source position detection unit 102 detects the positions of all light sources including the visible light communication light source in the second captured image. Then, it is determined whether each detected light source is a light source whose light intensity varies. If the light intensity varies, the detected light source is determined as a visible light communication light source, and the detected light source Is obtained as position information of the visible light communication light source. The light source position detection unit 102 may detect the visible light communication light source by a method other than the above. For example, the visible light communication light source emits light of a predetermined color that is different from other light sources, and the second photographed image is searched for an area where pixels close to this color exist. You may make it detect the position which shows the performed area | region as a position of the area | region of a visible light communication light source. The light of a predesignated color includes light from which a color of a specific wavelength is excluded. In addition, a marker indicating that the visible light communication light source has been specified in advance, on the surface of the visible light communication light source or in the vicinity thereof, is provided in advance, and the specified shape is obtained by pattern matching or the like. A process of detecting a matching shape may be executed in the second captured image, and a matching image or an area where pixels in the vicinity thereof have high luminance may be detected as a visible light communication light source. This marker is preferably a marker having a color that is easy to detect, such as a red LED. Alternatively, the shape or arrangement of the visible light communication light source is designated in advance, the shape matching the designated shape or arrangement is detected in the second captured image by pattern matching or the like, and the matching shape area May be determined to be a visible light communication light source. In the case where the visible light communication light source performs visible light communication by completely turning on and off the light, when a moving image is used as the second captured image used for detecting the position of the visible light communication light source, the first is not necessarily required. Since not all the frames of the two captured images include the image of the visible light communication light source, even if the detection of the visible light communication light source as described above is performed on the image of one frame, the visible light is not always sure. The communication light source cannot always be detected. Therefore, the visible light communication light source is detected with the color and shape as described above for each of the plurality of frames, and all the information on the position where the visible light communication light source is detected at least once is used as the position information of the visible light communication light source. It is preferable to obtain. That is, it is preferable to obtain the logical sum of the position information of the visible light communication light source acquired from each of the plurality of frame images. Alternatively, the position information of the visible light communication light source may be acquired from an image obtained by combining a plurality of frames of a moving image in advance. Moreover, when using a still image as a 2nd picked-up image, it is preferable to use the still image image | photographed with exposure time sufficiently longer than the cycle of the light source of visible light communication on and off. It is preferable that the light source position detection unit 102 continuously and repeatedly acquires the position information of the image of the visible light communication light source in the second captured image, and always updates the position information to the latest information. By doing in this way, even if the position of the imaging device 1 is changed due to movement or the like, it is possible to always obtain accurate position information of the visible light communication light source. The light source position detection unit 102 can be usually realized by an MPU, a memory, or the like. The processing procedure of the light source position detection unit 102 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  In the present embodiment, in particular, the light source position detection unit 102 includes a light source detection unit 1021 and a light source selection unit 1022, which are used to acquire the position information of the visible light communication light source from the second captured image. An example of the case will be described.

  Here, as an example, from the visible light communication light source, transmission target information that is information to be transmitted by visible light communication and a marker signal that is a signal used to detect the position of the visible light communication light source It is assumed that the information superimposed with is transmitted. The transmission target information described here is, for example, information related to a subject that is captured by the first imaging unit 100. Specifically, the transmission target information may be any information regarding the subject, and may be any information such as image data, audio data, text data, and an execution file. The transmission target information is, for example, information on the name, outline, detailed description, specifications, functions, operations, features, etc. of the subject. Further, it may be image information related to the subject, information such as an expected map, and the like. The subject may be any object or landscape. The marker signal is a signal having a predetermined frequency and a value whose value fluctuates in a predetermined pattern. The marker frequency is usually binary data in which a predetermined pattern is repeated at a predetermined cycle. Here, the frequency of the marker signal is referred to as a marker frequency. It is assumed that the marker frequency is a frequency at which the second image sensor 1011 can acquire information. On the other hand, the transmission target information transmitted by visible light communication may be information having a frequency higher than the frequency receivable by the second image sensor 1011 as long as the light receiving unit 1033 described below can receive light. In addition, here, the light output from the visible light communication light source transmits information due to fluctuations in the intensity of light, and is normally turned on when information is transmitted.

  The light source detection unit 1021 detects a light source in a second captured image that is an image acquired by the second imaging unit 101. The light source to be detected here is a light source including a visible light communication light source and a light source other than the visible light communication light source. Specifically, the light source detection unit 1021 detects a high luminance region from the second captured image captured by the second image sensor 1011. When the second captured image is a color image, the luminance described here may be a luminance for a specific color or color component, for example, a blue luminance. Further, a color region in a specific range may be detected. The areas detected in this way are areas determined to be all light sources including the visible light communication light source in the second photographed image. For example, the light source detection unit 1021 is a light source in an area where pixels having high luminance in the second captured image, for example, pixels having a luminance value equal to or higher than a threshold, exist adjacent to each other by a predetermined number of pixels. You may judge. The light source detection unit 1021 may detect the position of the light source by other methods. For example, when the area of a color specified in advance, for example, the color of light emitted from the light source is known in advance, by specifying this color, an area where pixels close to this color in the second photographed image are present can be obtained. It can be determined that the light source. Alternatively, when the shape of the light source can be specified in advance, the light source may be detected from the second captured image by pattern matching of the shape of the light source. The light source detection unit 1021 can be usually realized by an MPU, a memory, or the like. The processing procedure of the light source detection means 1021 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The light source selection unit 1022 selects a visible light communication light source in the second photographed image from among the light sources detected by the light source detection unit 1021, and acquires position information of the selected visible light communication light source. Normally, the visible light communication light source changes the light intensity in a short time to perform visible light communication, but other light sources often have almost no light intensity fluctuation. Therefore, assuming that the light source other than the visible light communication light source is a light source whose light intensity does not vary with time, such as an LED light source, the light intensity of the light source captured in the second captured image is The changing light source becomes a visible light communication light source. Therefore, for example, the light source selection unit 1022 obtains position information of an image of a visible light communication light source that is an image of a light source whose light intensity varies with time from among the light source images detected by the light source detection unit 1021. May be.

  Here, the response speed of the second image sensor 1011 which is an image sensor such as a CCD or CMOS is slower than that of a photo detector or the like, and the frequency of the electric signal detectable from the electric signal output from the second image sensor 1011 is currently set. About 100 KHz is the highest, and it is impossible to detect a fluctuation of an electric signal having a frequency of about 100 MHz as in the case of using a photodetector with a fast response speed. Accordingly, in order to increase the bandwidth of visible light communication, when transmitting transmission target information modulated at a high frequency by visible light communication, only the transmission target information is transmitted using the second image processing means 1012. It is difficult to detect fluctuations in light intensity from the light of visible light communication. For this reason, the marker signal described above having a relatively low frequency is superimposed on information to be transmitted by visible light communication and transmitted from the visible light communication light source. This marker signal can also be detected from an electrical signal output from the second image sensor 1011 having a slow response speed. For this reason, here, the light source selection unit 1022 separates a marker frequency signal, that is, a signal having the same frequency as the marker signal, from the electrical signal output from the second image sensor 1011 according to the light transmitted from each light source. By taking out and determining whether this signal is a marker signal whose value is fluctuating, it is determined whether the light source is a visible light communication light source that transmits information superimposed with the marker signal by visible light communication To do.

  Specifically, the light source selection unit 1022 includes a light receiving element (not shown) included in an area in the second imaging element 1011 corresponding to one area among areas determined by the light source detection unit 1021 as a light source. However, the electric signal output according to the incidence of light is continuously acquired for a predetermined time. The electrical signal output by the light receiving element included in the area corresponding to the light source in the second image sensor 1011 is included in the area corresponding to the light source among the plurality of light receiving elements constituting the second image sensor 1011. A value obtained by adding the electric signals output from the light receiving elements described above may be used, or an average value of the electric signals output from the plurality of light receiving elements may be used. The light source selection unit 1022 extracts marker frequency information from the electrical signal continuously acquired from one region. At this time, if one region is a visible light communication region, the obtained electrical signal is a signal of information obtained by superimposing transmission target information and a marker signal as described above.

  The light source selection unit 1022 acquires a predetermined number of voltage values at a predetermined sampling period from the extracted marker frequency information, and determines whether or not the acquired voltage values have changed. For example, it is determined whether the difference between the maximum value and the minimum value of the acquired voltage is equal to or greater than a threshold value. If the difference is equal to or greater than the threshold value, it is determined that fluctuation has occurred. Judge. When it is determined that fluctuation has occurred, for example, when the difference between the maximum value and the minimum value is greater than or equal to the threshold value, it can be considered that the electrical signal acquired from one region includes a marker signal. , It is determined that one region is a light source for visible light communication. Conversely, if no fluctuation occurs, it is determined that the light source is another light source whose light intensity does not change. Then, the same processing is performed for other regions that the light source detection unit 1021 determines to be light sources, thereby determining whether each light source captured in the second captured image is a visible light communication light source. be able to. Then, the light source selection unit 1022 acquires information indicating the position of the light source determined to be the visible light communication light source in the second captured image as the position information of the visible light communication light source. In addition, instead of acquiring information indicating the position of the light source determined to be the visible light communication light source in the second captured image as the position information of the visible light communication light source, visible light communication in the second image sensor 1011 is performed. Information indicating the position of a region that receives light incident from a light source determined to be a light source may be acquired as position information of the visible light communication light source.

  In order to reliably detect the marker signal by a predetermined number of times of sampling, the frequency and pattern of the marker signal may be determined based on the relationship between the marker signal and the sampling period as described below, for example.

  FIG. 4 is a schematic diagram showing the relationship between the marker signal and the sampling period. In the figure, a sampling period 41 indicates the timing for sampling. Here, sampling is performed at regular time intervals. The marker signals 42a to 42j are signals having binary values “H” and “L”, respectively. Here, “H” is indicated by diagonal lines, and “L” is indicated by plain color. Here, assuming that the sampling interval indicated by the sampling period 41 is “1”, the period of the marker signals 42a to 42j and the ratio of the period in which the binary value that is the pattern of the marker signals 42a to 42j appears is represented. For example, the marker signal 42a is a signal whose frequency is 1/3 of the sampling period 41, the period of “H” is 1/3, and the period of “L” is 2/3. The marker signal 42b is a signal whose frequency is 1/3 of the sampling period 41, the period of “H” is 1/3, and the period of “L” is 1/3. The marker signal 42c is a signal whose frequency is ½ of the sampling period 41, the “H” period is ½, and the “L” period is 1.

  For example, in the marker signal 42a, when sampling is performed at the sampling period 41, only one data is always detected, for example, “H” only. For this reason, when the marker signal 42a is used, it is not possible to detect a marker signal that is a signal whose value varies. On the contrary, in the marker signal 42b, when sampling is performed at the sampling period 41, signals of “H” and “L” are detected alternately. Therefore, when the marker signal 42a is used, when the signal extracted at the marker frequency is a signal whose value fluctuates between “H” and “L”, the marker signal is detected. Therefore, the light source position detection unit 102 can determine whether the light source is a visible light communication light source. In order to reliably detect the marker signal by sampling a predetermined number of times, the sampling period may be changed. The light source selection unit 1022 can usually be realized by an MPU, a memory, or the like. The processing procedure of the light source selection unit 1022 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  When the image captured by the second imaging unit 101 is a moving image composed of a plurality of frame images, first, the light source detection unit 1021 first frame of the second captured image captured by the second image sensor 1011. From the image, a region having a high luminance is detected, and for a plurality of subsequent frame images, it is determined whether or not the luminance value acquired for each detected high luminance region has a variation greater than a predetermined threshold value, When the fluctuation has occurred, it may be determined that one region is a light source for visible light communication in which the light intensity changes. In addition, when detecting the position of the visible light communication light source from the frame image in this way, as in the above, the change in the luminance of the visible light communication light source can be reliably detected using a plurality of frame images. It is necessary to set the pattern and frequency at which the value changes according to the frame rate of the second captured image and the like. In this case, it may be considered that the frame rate is the sampling rate described above.

  The information acquisition unit 103 acquires communication light incident position information, which is information on a position where light output from the visible light communication light source is incident, from the position information of the visible light communication light source acquired by the light source position detection unit 102. Transmission target information, which is information transmitted by visible light communication from the visible light communication light source at the position indicated by the communication light incident position information, is acquired. Specifically, the information acquisition unit 103 indicates a correspondence relationship between positions where light that has passed through the second lens 201 is incident on the information acquisition unit 103 and captured images obtained by capturing light that has passed through the second lens 201. Information is stored in a storage medium or the like in advance, and the information acquisition unit 103 corresponding to the position information of the visible light communication light source in the captured image acquired by the light source position detection unit 102 using the information indicating the correspondence relationship. Information on a position on a region where light having passed through the second lens 201 is incident is acquired. Then, only light incident on the acquired position is selectively received, and transmission target information is acquired from the received light. Specifically, the region where light having passed through the second lens 201 of the information acquisition unit 103 is incident is a region where light on a liquid crystal panel 10321 described later is incident. In addition, the information acquisition unit 103 receives light output from the visible light communication light source every time the light source position detection unit 102 acquires the position information of the visible light communication light source or the position information changes, for example. Communication light incident position information, which is information indicating the position, is acquired, and transmission target information transmitted from the visible light communication light source is acquired at the position indicated by the communication light incident position information.

  The information acquisition unit 103 is transmitted from the visible light communication light source by visible light communication in synchronization with the image capturing performed by the first image capturing unit 100 in order to acquire timely information at the time of image capturing by visible light communication. It is preferable to acquire the transmission target information. Acquiring in synchronization with the shooting described here means acquiring information transmitted at the time of shooting or at timings before and after the time of shooting. Specifically, the information acquisition unit 103 recognizes the start point and the end point of visible light communication information transmitted by visible light communication at the time when the first image capturing unit 100 performs shooting, and from the recognized start point to the end point. The visible light communication information in the range is acquired by determining as transmission target information transmitted in synchronization with imaging. The period described here is the period before and after shooting. The start point or end point here indicates information indicating the start point of one information, for example, a start code or the end point of one information, included in information received from visible light communication at or before and after the time when shooting was performed. Information, for example an exit code. For example, the information acquisition unit 103 acquires information in a range from information indicating a start point that appears at the closest time during photographing to information indicating an end point that appears next in information transmitted by visible light communication. . This acquired information is information acquired in synchronization with photographing.

  When the light source position detection unit 102 acquires communication light incident position information corresponding to two or more visible light communication light sources, the information acquisition unit 103 receives two or more visible light communication light sources from a reception unit (not shown). Receiving a selection instruction, which is an instruction to select any one of the position information, obtaining communication light incident position information corresponding to the position information of the visible light communication light source specified by the selection instruction, and entering the communication light You may make it acquire the transmission object information transmitted by visible light communication from the visible light communication light source in the position which position information shows. Further, communication light incident position information corresponding to position information of the visible light communication light source having the largest area in the second captured image may be acquired. Alternatively, the transmission target information may be alternately acquired at a specific timing or an unspecified predetermined timing at two or more positions indicated by communication light incident positions corresponding to two or more visible light communication light sources.

  Here, the information acquisition unit 103 includes an incident position information acquisition unit 1031, a selective transmission unit 1032, a third lens 203, a light receiving unit 1033, and an information acquisition unit 1034, which are used to generate a visible light communication light source. The process of selectively receiving the transmitted light and acquiring the transmission target information transmitted from the visible light communication light source will be described below.

  The incident position information acquisition unit 1031 is a visible light communication light source in a region where light output from the visible light communication light source of the information acquisition unit 103 is incident based on the position information of the visible light communication light source acquired by the light source position detection unit 102. The communication light incident position information, which is information on the position where the light output from is incident, is acquired. Specifically, the second photographed image photographed by the second photographing unit 101 and the shape of the region where the light that has passed through the second lens 201 in the information acquiring unit 103 is incident are similar, and the coordinate axes of both are In the case where the same direction is set with respect to the image incident through the second lens 201, the second photographed image is obtained by using the similarity ratio between the second photographed image and the light incident region. Is the coordinate information of the position where the light from the visible light communication light source is incident in the region where the light passing through the second lens 201 in the information acquisition unit 103 is incident from the position information which is the coordinate information of the visible light communication light source. Acquire communication light incident position information. For example, when the position information is coordinate information, when the similarity ratio is 1: 1, the incident position information acquisition unit 1031 has coordinates of the visible light communication light source in the second captured image acquired by the light source position detection unit 102. Information is acquired as coordinate information, which is information of an incident position in the region where light is incident in the information acquisition unit 103 as it is. Further, when the similarity ratio between the second photographed image and the light incident region is 2 to 1, the values of the x-axis and y-axis of the coordinate information of the visible light communication light source in the second photographed image are respectively divided into two. Is acquired as coordinate information that is information on an incident position in a region in which light is incident in the information acquisition unit 103. That is, when the similarity ratio between the captured image and the light incident area is n (n is an arbitrary positive number) to 1, the x-axis and y-axis of the coordinate information of the visible light communication light source in the captured image The coordinate information whose value is 1 / n is acquired as coordinate information that is information on the incident position in the region where light is incident in the information acquisition unit 103. In addition, information that associates one or more pixels in the second captured image with information indicating the position of a region where light is incident in the information acquisition unit 103, for example, coordinate information, is managed in advance. From the attached information, information indicating the position in the region where light is incident in the information acquisition unit 103 corresponding to the pixel in which the visible light communication light source is detected in the second captured image is acquired as communication light incident position information. Anyway. In the present embodiment, as will be described later, a case where the region where the light that has passed through the second lens 201 of the information acquisition unit 103 is incident is a liquid crystal panel 10321 will be described as an example. For this reason, instead of using the coordinate information in the liquid crystal panel 10321 as the communication light incident position information, information specifying a pixel (not shown) in the liquid crystal panel 10321, for example, an ID or a number of the pixel may be used. . For example, information that associates one or more pixels in the second captured image with one or more pixels of the liquid crystal panel 10321 is managed in advance, and the visible information in the second captured image is determined based on the associated information. Information specifying the pixel of the liquid crystal panel 10321 corresponding to the pixel from which the optical communication light source is detected may be acquired. Note that if the second captured image captured by the second imaging unit 101 and the shape of the region where the light that has passed through the second lens 201 in the information acquisition unit 103 is incident are not completely similar, depending on the difference in shape The position information to be acquired may be corrected. The position information of the visible light communication light source acquired by the light source position detection unit 102 is, for example, the position information of a part of the visible light communication light source such as the position information of the center position of the visible light communication light source in the second captured image. In this case, the incident position information acquisition unit 1031 sets the area including the area around the position corresponding to the position information in the area where the light is incident in the information acquisition unit 103 as the communication light incident position. You may do it. The incident position information acquisition unit 1031 can be usually realized by an MPU, a memory, or the like. The processing procedure of the incident position information acquisition unit 1031 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The selective transmission means 1032 selectively transmits only the light incident on the position indicated by the communication light incident position information acquired by the incident position information acquisition means 1031 among the light incident through the second lens 201. Specifically, the selective transmission means 1032 has a filter, a diaphragm, or the like that can form a region that can transmit light only in an arbitrary region within the region where light having passed through the second lens 201 is incident. By controlling the filter and the diaphragm, an area where light can be transmitted is formed only at the position indicated by the incident position information acquired by the incident position information acquiring unit 1031. Here, a case where the selective transmission means 1032 includes the liquid crystal panel 10321 and the light shielding control means 10322 and the above processing is performed using these will be described.

  The liquid crystal panel 10321 changes the orientation of the liquid crystal by applying a voltage to the liquid crystal with a plurality of arranged electrodes, and controls whether to transmit light for each segment provided with the electrode, thereby transmitting light. The shape of the area to be controlled can be controlled. Here, for example, a liquid crystal panel in which segments are arranged in a lattice shape is used. This segment is called a pixel here. The liquid crystal panel 10321 may be a color liquid crystal panel such as an RGB color, or a binary or gray scale liquid crystal panel such as black and white. The liquid crystal panel 10321 only needs to be able to display two colors, a color that allows light to pass through and a color that does not allow light to pass, and the number of gradations that can be displayed only needs to be two or more. By using a color liquid crystal panel as the liquid crystal panel 10321, the color of the region through which light is transmitted can be set to an arbitrary color, for example, blue. By doing in this way, it becomes possible to permeate | transmit only the arbitrary colors of the light from a visible light communication light source, for example, only blue. The liquid crystal panel 10321 is disposed at a position where light having passed through the second lens 201 is incident.

  The light blocking control unit 10322 controls the liquid crystal panel 10321 so that a region other than the incident position indicated by the incident position information acquired by the incident position information acquiring unit 1031 of the liquid crystal panel 10321 is a region that blocks light. In this case, the incident position information acquired by the incident position information acquiring unit 1031 is preferably information that can specify one or more pixels of the liquid crystal panel 10321. Specifically, a control signal for controlling each pixel of the liquid crystal panel 10321 is output to the liquid crystal panel 10321, and the pixel of the liquid crystal panel 10321 corresponding to the communication light incident position acquired by the incident position information acquisition unit 1031 transmits light. The liquid crystal panel 10321 is controlled so as to be a possible pixel and pixels in other regions to be pixels that block light. As a result, only the light input to the communication light incident position out of the light incident on the liquid crystal panel 10321 through the second lens 201, that is, only the light from the visible light communication light source is transmitted through the liquid crystal panel 10321. Become. A transmissive pixel is a pixel that can transmit light from visible light communication. The transmissive pixel may be, for example, a pixel that is transparent or translucent, or may be a pixel that transmits only a specific color of light. For example, when the liquid crystal panel 10321 is a color liquid crystal panel, the pixel corresponding to the communication light incident position acquired by the incident position information acquisition unit 1031 of the liquid crystal panel 10321 is set as a blue pixel, and the other area is set as black or the like. Thus, only blue light can be selectively transmitted from the pixel position corresponding to the communication light incident position. The shading control means 10322 can be usually realized by an MPU, a memory, or the like. The processing procedure of the shading control means 10322 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The third lens 203 is a condensing unit that condenses the light transmitted through the selective transmission unit 1032, specifically, the liquid crystal panel 10321 and causes the light to enter the light receiving unit 1033. The third lens 203 needs to be installed so that light transmitted through the selective transmission means 1032, specifically, any position of the liquid crystal panel 10321 is incident on the light receiving means 1033. Instead of the third lens 203, other condensing means such as a concave mirror may be provided. Further, when the light transmitted through the selective transmission unit 1032 can be incident on the light receiving unit 1033 without being condensed, such as when the size of the light receiving unit 1033 is sufficiently large, the third lens 203 can be omitted. is there.

  The light receiving means 1033 receives light incident on the incident position indicated by the incident position information and converts it into an electrical signal. Specifically, light selectively extracted by the selective transmission means 1032, specifically, the liquid crystal panel 10321 is condensed by the third lens 203 and is incident on the light receiving means 1033. The light receiving means 1033 converts the intensity of the incident light in this way into an electric signal. The light receiving means 1033 is a light receiving element having a high response speed, specifically a photodetector. The light receiving means 1033 may be composed of a plurality of photodetectors as long as it functions as a single photodetector. By using one photodetector as the light receiving means 1033, light can be converted into an electrical signal at high speed, and visible light communication light that fluctuates at high speed can be received with high accuracy.

  The information acquisition unit 1034 acquires information transmitted by visible light communication from the electrical signal converted by the light receiving unit 1033. This information is transmission target information. For example, the information acquisition unit 1034 demodulates the electrical signal converted by the light receiving unit 1033 at a predetermined frequency specified in advance, and extracts information. Further, AD conversion or the like may be appropriately performed on the extracted information. In addition, as described above, the information acquisition unit 1034 preferably acquires transmission target information transmitted from the visible light communication light source through visible light communication in synchronization with the image capturing performed by the first image capturing unit 100. . Specifically, the information acquisition unit 1034 recognizes the start point and the end point of the transmission target information transmitted by visible light communication at the time when the first image capturing unit 100 performs image capturing, and the information from the recognized start point to the end point. It is preferable to acquire the transmission target information in the range by determining that the transmission target information is transmitted in synchronization with imaging. The configuration and the like of the information acquisition unit 1034 are the same as the information acquisition unit and the like used in a receiving terminal of a normal visible light communication system, and thus detailed description thereof is omitted. The information acquisition unit 1034 can usually be realized by an MPU, a memory, or the like. The processing procedure of the information acquisition unit 1034 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 104 outputs a first captured image that is an image acquired by the first imaging unit 100 and transmission target information acquired by the information acquisition unit 103. Specifically, the output unit 104 outputs the first captured image acquired by the first imaging unit 100 and the transmission target information acquired by the information acquisition unit 103 in association with each other. Here, the association may be to output information obtained by pairing the first captured image and the acquired transmission target information as long as the association can be determined. The first captured image and the acquired transmission target information may be output together with the management information indicating the correspondence relationship with the acquired transmission target information. Moreover, you may output the information which combined the 1st picked-up image and the acquired transmission object information, for example, the synthesized image information. The transmission target information acquired by the information acquisition unit 103 is transmission target information transmitted from the visible light communication light source by visible light communication. Specifically, the transmission target information is information related to a subject that is photographed by the first photographing unit 100. When the information acquisition unit 103 acquires transmission target information transmitted from the visible light communication light source through visible light communication in synchronization with the image capturing performed by the first image capturing unit 100 as described above, the output unit 104 The first captured image acquired by the first image capturing unit 100 and the information acquired by the information acquiring unit 103 in synchronization with the capturing of the first captured image are preferably output in association with each other. Thereby, the transmission target information transmitted by visible light communication at the time of imaging or before and after imaging can be output in a timely manner. For example, when different information is transmitted for each time zone with respect to the same subject by visible light communication, it is possible to acquire and output the transmission target information transmitted at the time of shooting. Output described here includes display on a display, printing on a printer, sound output, transmission to an external device, storage in a recording medium, delivery of processing results to another processing device or other program, etc. It is a concept that includes. The output unit 104 may or may not include an output device such as a display or a printer. The output unit 104 can be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the photographing apparatus 1 in the present embodiment will be described using the flowchart of FIG. Here, a case where information in which a marker signal having a different frequency and information to be subjected to visible light communication are superimposed is transmitted from the visible light communication light source by visible light communication will be described as an example. Information that is a target of visible light communication is information about a target of shooting by the first shooting unit 100.

  (Step S201) The second imaging unit 101 captures an image and acquires a second captured image. The second captured image may be a still image or a moving image composed of a plurality of frame images.

  (Step S202) The light source position detection unit 102 acquires the position information of the visible light communication light source in the second captured image acquired in Step S201. Details of this processing will be described later.

  (Step S203) The incident position information acquisition unit 1031 acquires communication light incident position information corresponding to the position information of the visible light communication light source acquired in Step S202. Specifically, using information indicating the correspondence between the captured image and the liquid crystal panel, the position information on the liquid crystal panel 10321 corresponding to the position of the visible light communication light source in the captured image is obtained from the position information of the visible light communication light source. get.

  (Step S204) The light blocking control unit 10322 controls the liquid crystal panel 10321 using the communication light incident position information acquired in step S203, and light is transmitted only through the pixel at the position indicated by the communication light incident position information of the liquid crystal panel 10321. As a possible state, pixels in other regions are made to block light. For example, the liquid crystal panel 10321 displays an image in which only the region indicated by the communication light incident position information is a transmissive region and the other regions are black that does not transmit light. As a result, only the light output from the visible light communication light source is transmitted through the region constituted by the pixel at the position indicated by the communication light incident position information of the liquid crystal panel 10321, and the other light is blocked by the liquid crystal panel 10321.

  (Step S205) The information acquisition unit 1034 starts acquisition of information transmitted by visible light communication from an electrical signal output by the light receiving unit 1033 that has received the light transmitted through the liquid crystal panel 10321. The acquired information is temporarily stored in a storage medium such as a delay buffer or a memory (not shown). It should be noted that information that has passed a fixed or indefinite predetermined time since acquisition may be discarded.

  (Step S206) The photographing apparatus 1 determines whether or not an instruction for photographing an image by the first photographing unit 100 has been received. For example, the first photographing unit 100 determines whether or not a trigger for performing photographing, such as an operation of pressing a shutter button (not shown) by the user, has been received. If accepted, the process proceeds to step S207, and if not accepted, the process proceeds to step S210.

  (Step S207) The first imaging unit 100 captures an image and obtains a first captured image. The first captured image may be a still image or a moving image composed of a plurality of frame images.

  (Step S208) The information acquisition unit 103 acquires information transmitted by optical wireless communication in synchronization with imaging performed by the first imaging unit. Specifically, in step S205, information indicating the starting point of one piece of information designated in advance, such as a start code, is detected from the information temporarily stored in the storage medium or the like, and the start code is detected. Start acquiring information from a location. And the information which shows the end point designated next beforehand among the information transmitted by visible light communication, for example, information until an end code is detected is acquired. If the information temporarily indicating in step S205 does not include the information indicating the start point, the information indicating the start point is detected from the newly received information, and the information from the position where the information indicating the start point is detected is detected. You only need to get it. The acquired information is stored in a storage medium such as a memory (not shown).

  (Step S209) The output unit 104 outputs the first captured image acquired in step S207 and information synchronized with the imaging acquired in step S208. For example, these pieces of information are associated and stored in a storage medium such as a memory (not shown).

  (Step S210) The information acquisition unit 103 determines whether to stop receiving information transmitted by visible light communication. It may be determined to stop the output in response to any trigger. For example, when an instruction to stop reception is received from an unillustrated reception unit or the like, it may be determined to stop reception, or when an instruction to turn off the power of the photographing apparatus 1 is received, or when an operation is performed If the photographing apparatus 1 enters a standby state after a predetermined period of time has elapsed, it may be determined to stop reception. If the reception is to be canceled, the process is terminated. If the reception is not to be canceled, the process returns to step S201.

  If it is determined in step S208 that the reception is not stopped, the same communication light incident position information is returned to step S205 until a predetermined time elapses after the position of the visible light communication light source is acquired. The transmission target information may be repeatedly acquired at the position indicated by, and when a predetermined time has elapsed, the process may return to step S201 to acquire the position of the visible light source again.

  In addition, when it is determined in step S210 that the reception is not stopped, the process returns to step S201. However, the information acquisition unit 103 started in step S205 is visible even during the subsequent processing after step S201. You may make it perform the process which receives the information transmitted by optical communication in parallel.

  Note that immediately after step S202, the incident position information acquisition unit 1031 may determine whether or not the position information of a plurality of visible light communication light sources has been acquired in step S202. Then, when the position information of a plurality of visible light communication light sources is acquired, the position information of one visible light communication light source is selected from the plurality of visible light communication light sources, and the selected visible light communication is performed. Communication light incident position information corresponding to the position information of the light source may be acquired. For selecting the position information of the visible light communication light source, for example, an instruction to select one of the visible light communication light sources from the user is accepted, and the position information of the visible light communication light source corresponding to the instruction is selected. Also good. Moreover, you may make it select the position information of any one visible light communication light source according to the rule designated beforehand. This rule may be any rule, for example, position information may be selected at random, or position information closer to the center of the captured image may be selected. Moreover, you may make it select the one where the intensity | strength of the light obtained from each positional information is higher. Further, the position information of the visible light communication light source having the largest size may be selected. If position information of a plurality of visible light communication light sources has not been acquired, the process may proceed to step S203.

  Next, details of a process of acquiring the position information of the visible light communication light source using the above-described second captured image will be described using the flowchart of FIG. This process is a process corresponding to the process of step S202.

  (Step S301) The light source detection unit 1021 detects a light source in the second captured image acquired in Step S201 described above. As a specific example, since the light source is usually composed of pixels having a high luminance value in the second photographed image, the luminance value and the pixels constituting the light source are determined for each pixel constituting the second photographed image. The threshold value for determination is sequentially compared. Then, a pixel having a luminance value equal to or higher than the threshold is determined to be a light source pixel. A region in which pixels determined to be light source pixels continuously exist is sequentially detected as one light source. Then, the coordinate information of each pixel constituting the light source is stored in a storage medium (not shown) in association with identification information for identifying the light source, for example. Here, a set of coordinate information of one or more pixels constituting one light source is set as position information indicating the position of the one light source. It should be noted that the center coordinates, the barycentric coordinates, the coordinates defining the contour, and the like of the area constituting one light source may be used as the light source position information. The position of the light source may be detected by other methods. For example, when the shape of the light source is determined in advance, it may be detected by performing pattern matching on the shape of the light source. Note that the second photographed image used here may be a still image or one frame image in a moving image, or may be an image obtained by combining a plurality of moving image images. In addition, here, the luminance value of the second captured image is used, but the light source may be detected by the color in the second captured image, or the value of a specific channel in the second captured image, For example, the light source may be detected using values of the R channel, the G channel, and the B channel.

  (Step S302) The light source selection unit 1022 substitutes 1 for the counter q.

  (Step S303) The light source selection unit 1022 determines whether or not the qth light source is present in the light sources detected in step S301. If there is, the process proceeds to step S304. If not, the acquired information on the position of the visible light communication light source is returned to the higher-level process.

  (Step S304) The light source selection unit 1022 acquires an electrical signal obtained at a position of the second image sensor 1011 corresponding to the qth light source. Specifically, the light source selection unit 1022 is an electrical signal output from one or more light receiving elements arranged at a position corresponding to the position of the qth light source in the second captured image of the second image sensor 1011. To get. In addition, since a 2nd picked-up image is created from the electric signal which each light receiving element of the 2nd image pick-up element 1011 outputs, the position of each pixel in a 2nd picked-up image and the electric signal used in order to create the said pixel Since the correspondence relationship with the position of one or more light receiving elements that output is specified in advance, the information indicating the correspondence relationship is used to determine the position of the light receiving element corresponding to the pixel constituting the qth light source. It is possible to obtain. Here, unlike the case of shooting, the electric signal output from the second image sensor 1011 is continuously acquired without scanning the second image sensor 1011. Thereby, the light output from the q-th light source can be converted into a continuous electric signal.

  (Step S305) The light source selection unit 1022 acquires a marker frequency signal, which is the frequency of the marker signal, from the electrical signal acquired from Step S303. In addition, since the technique which takes out the signal of a predetermined frequency from an electrical signal is a well-known technique, description is abbreviate | omitted here.

  (Step S306) The light source selection unit 1022 samples, or digitizes, the electrical signal acquired in step S305 at a predetermined cycle specified in advance.

  (Step S307) The light source selection unit 1022 determines whether or not a predetermined time has elapsed from the start of the process of step S304 for the qth light source. Specifically, the predetermined time is information that specifies a period during which a signal for detecting a marker signal is received from one light source. If the predetermined time has elapsed, the process proceeds to step S308. If not, the process returns to step S303. In addition, after receiving the electrical signal output from the second image sensor 1011 for a predetermined time, the received electrical signal may be sampled.

  (Step S308) The light source selection unit 1022 detects whether or not the signal acquired at the marker frequency is a marker signal. The detection of the marker signal here may be considered as detecting the detection of a signal whose signal magnitude is characteristic of the marker signal. Specifically, in step S306, the light source selection unit 1022 determines whether or not the intensity value of the electrical signal acquired by sampling within the predetermined time described above has fluctuated. It is determined that the signal is a marker signal, and if it is not changed, it is determined that the signal is not a marker signal. Whether or not it fluctuates may be determined in any way. For example, it fluctuates when the difference between the maximum value and the minimum value of values sampled within a predetermined time is larger than a preset threshold value. You may judge that Alternatively, it may be determined that the sampled value fluctuates when at least one value greater than a preset maximum value and one less than a preset minimum value are included. When the marker signal is detected, the process proceeds to step S309, and when the marker signal is not detected, the process proceeds to step S310.

  (Step S309) The light source detection unit 1021 acquires information on the position of the q-th light source among the light sources detected in Step S301 as position information of the visible light communication light source. For example, the coordinate information of one or more pixels constituting the qth light source is acquired as the position information of the visible light communication light source. Then, the acquired position information is temporarily stored in a storage medium (not shown) or the like in association with, for example, information for identifying the visible light communication light source. The temporarily stored position information of the visible light communication light source is the position information of the visible light communication light source acquired by the light source position detection unit 102.

  (Step S311) The light source selection unit 1022 increments the value of the counter q by 1. Then, the process returns to step 303.

  In the flowchart described with reference to FIG. 2, once the position of the visible light communication light source is detected, the second captured image is repeatedly captured by the second capturing unit 101 as shown in step S201, and the newly captured second captured image is acquired. The position of the light source is newly detected from the brightness of the pixel or the like in the same manner as the processing in step S301 described above, and the newly detected light source has a high correlation with the visible light communication light source detected immediately before, or You may make it detect the light source etc. with the shortest distance with the visible light communication light source detected immediately before as a position of the visible light communication light source in the newly image | photographed image. The visible light communication information may be received using the detected position information of the visible light communication light source. As a result, even when the position of the visible light communication light source changes, it is possible to detect the position after the change at high speed and to receive information communicated by visible light following the change in position.

  Hereinafter, a specific operation of the photographing apparatus 1 in the present embodiment will be described. Here, a case will be described in which the photographing apparatus 1 constitutes at least a part of the portable terminal 1a. The mobile terminal 1a is specifically a mobile phone. However, other portable terminals such as a PDA, a portable music player, and a portable video player may be used.

  FIG. 5 is a front perspective view of the portable terminal 1 a provided with the photographing apparatus 1.

  FIG. 6 is a rear perspective view of the portable terminal 1 a provided with the photographing device 1.

  The portable terminal 1a includes a first lens 200 on the back surface. Further, the mobile terminal 1a includes a second lens 201 on the upper surface, and when the first lens 200 of the mobile terminal 1a is directed toward the subject so that the optical axis thereof is substantially horizontal, the light of the second lens 201 is obtained. It is designed so that the axis is almost upward. It is assumed that the portable terminal 1a includes a display device 1051 that is an output device on the front surface.

  FIG. 7 is a schematic diagram illustrating a usage example of the mobile terminal 1a. Here, as an example, the room where the portable terminal 1a exists is assumed to be a museum exhibition room where photography is permitted. It is assumed that a plurality of LED light sources 51 to 56 are installed on the ceiling of the room. In the vicinity of the light source 52 and the light source 55, it is assumed that an exhibit 71 and an exhibit 72 are arranged, respectively. Here, in particular, it is assumed that the light source 52 and the light source 55 are visible light communication light sources that modulate visible information and transmit the information transmitted from a transmission device (not shown) by power line communication (PLC) or the like to visible light. It is assumed that the distance between the light source 52 and the light source 55 is wider than the range that can be photographed at the angle of view of the second lens 201. Information relating to the exhibit 71, specifically, the name of the exhibit 71, text information explaining what the exhibit 71 is, and the like are transmitted from the light source 52 by visible light communication. Shall. Similarly, from the light source 55, information regarding the exhibit 72, specifically, the name of the exhibit 72, text information explaining what the exhibit 72 is, etc. by visible light communication, etc. Is sent. Information related to the exhibit 71 and the exhibit 72 is referred to herein as exhibit related information. The exhibit related information may be, for example, information composed of an image or sound other than text. Here, it is assumed that marker signals used for detecting the position of the visible light communication light source are transmitted from the light source 52 and the light source 55 in a manner superimposed on the above-described exhibit related information. As an example, it is assumed that the frequency of the marker signal is 28.8 KHz and the frequency at which each exhibition related information is transmitted is 80 MHz. In addition, the frequency of each exhibit related information may be a different frequency. Further, each light source described in this specific example may be an aggregate of a plurality of light sources as long as each substantially functions as one light source.

  First, in order for the user to take an image of the exhibit 71, the user stands in front of the exhibit 71 and holds the portable terminal 1a with the first lens 200 provided on the back of the portable terminal 1a facing the exhibit 71. To do. At this time, if the optical axis of the first lens 200 is substantially horizontal, the second lens 201 provided on the upper surface of the portable terminal 1a faces upward, that is, the ceiling of the exhibition room. It is assumed that the user turns on the photographing function of the mobile terminal 1a by operating a key or the like.

  The second imaging unit 101 of the imaging device 1 captures an image of the ceiling via the second lens 201. Here, the number of pixels of the image captured by the second imaging unit 101 is assumed to be 320 × 240 pixels as an example for explanation. Note that the greater the number of pixels, the more accurately the light source is detected, but the processing is slowed accordingly.

  FIG. 8 is a diagram illustrating a second captured image captured by the second imaging unit 101. Here, it is assumed that the second photographed image is a still image. In the second photographed image, light sources 51 to 53 that are lit and are arranged in the upper periphery of the exhibit 71 are photographed. In the figure, the same reference numerals as those in FIG. 7 indicate images of the same light source. In the image of FIG. 8, the upper left is the coordinate (0, 0), the right direction of the coordinate is the x-axis direction, and the lower direction of the coordinate is the y-axis direction. The unit of coordinates is assumed to be pixels.

  Next, the light source detection unit 1021 acquires the second captured image illustrated in FIG. 8 and detects the light source in the acquired second captured image. Here, the luminance value of each pixel of the second photographed image is compared with a threshold value of a luminance value specified in advance, and a pixel having a luminance value equal to or higher than the threshold value is determined as a light source pixel. Here, it is assumed that the luminance is higher as the luminance value is larger. Then, it is determined that a region where luminance values equal to or greater than the threshold value are continuous is one light source.

  For example, in the second photographed image shown in FIG. 8, a region where pixels having luminance values equal to or higher than the threshold value are white, and a region where pixels having luminance values smaller than the threshold value are indicated by hatching is shown in FIG. become. Through the process of detecting the light source as described above, it is determined that the three independent regions 51a to 53a represented in white in FIG. 9 are the light sources. The regions 51a to 53a are regions corresponding to the light sources 51 to 53, respectively.

  The detected coordinate information of the pixels constituting each light source is temporarily stored in a storage medium such as a memory (not shown) together with identification information given to each light source.

  FIG. 10 is a light source coordinate management table for managing the light source and the coordinate information of the pixels constituting the light source. The light source coordinate management table has an item “light source ID” which is identification information given to the light source and an item “coordinate” which is the coordinates of the pixels constituting the light source. Here, it is assumed that “001” to “003” are associated as “area IDs” with the pixels constituting the areas 51a to 53a, respectively.

  Next, the light source selection unit 1022 first outputs an electric signal output by the light receiving element in the region in the second image sensor 1011 corresponding to the region where the “light source ID” of the captured image is “001”, that is, the region 51a. Receive. For example, when the second image sensor 1011 and the captured image have a similar relationship, the region on the second image sensor 1011 that has a similar relationship with the region where the “light source ID” of the captured image is “001” Only the electric signal output according to the intensity of the light received by the light receiving element is continuously received.

  The light source selection unit 1022 separates the frequency of the marker signal, here a signal of 28.8 KHz, from the received electrical signal. Then, the separated signal is sampled at a predetermined sampling period. In this specific example, the relationship between the sampling period and the marker signal is such that when the marker signal is included, the marker signal can be reliably detected as described with reference to FIG. Keep it.

  The light source selection unit 1022 ends the reception and sampling of the electrical signal when a predetermined time, for example, 1/10 to 1/30 second, has elapsed from the start of the reception of the electrical signal, and the marker signal is received from the electrical signal. It is determined whether or not it is detected, that is, whether or not the signal separated from the electrical signal at the frequency of the marker signal is a marker signal whose value varies. For example, it is determined whether or not two values “H” and “L” are included in a value obtained by binarizing a plurality of values obtained by sampling with a predetermined threshold value. Here, since the light source 51 is not a visible light communication light source and the marker signal is not included in the electrical signal, the sampled value does not include the binary value. The light source selection unit 1022 determines that the marker signal is not detected because the sampled value does not include the binary value, and the position information of the region where the “light source ID” of the captured image is “001” is visible light. Not acquired as position information of the communication light source.

  Next, similarly, the light source selection unit 1022 outputs an electrical signal output by the light receiving element in the area in the second image sensor 1011 corresponding to the area where the “light source ID” of the captured image is “002”, that is, the area 52a. Receive. And the signal of the frequency of a marker signal is isolate | separated from this electric signal, and it samples. Then, similarly to the above, it is determined whether or not the sampled value is a marker signal. Here, since the light source 52 is a visible light communication light source, and the marker signal is included in the electrical signal, the sampled value includes two values. The light source selection unit 1022 determines that the marker signal is detected because the sampled value includes two values. Then, the light source selection unit 1022 uses the information indicating the position of the area where the “light source ID” of the captured image is “002”, here the coordinate information of the pixels included in the area 52a as an example, and the position information of the visible light communication light source. And stored in a memory or the like (not shown).

  The light source selection unit 1022 performs the same process for the region where the “light source ID” is “003”. Here, since the light source 53 is not a visible light communication light source, a marker signal is not detected from the region where the “light source ID” is “003” as in the case of the light source 51. For this reason, the light source selection unit 1022 does not acquire the position information of the region whose “light source ID” is “003” as the position information of the visible light communication light source.

  FIG. 11 is a visible light communication light source management table for managing position information of visible light communication light sources. The visible light communication light source management table has an item “light source ID” that is identification information of a visible light communication light source and an item “position information” that is coordinate information of each pixel constituting the visible light communication light source. . Here, “light source ID” corresponds to “light source ID” in the light source coordinate management table of FIG. 7, and light sources having the same value are assumed to be the same light source. Here, information indicating the position of the region having the “light source ID” “002” acquired by the light source selection unit 1022 is managed as the position information of the visible light communication light source.

  Next, using the position information of the visible light communication light source shown in FIG. 11, the incident position information acquisition unit 1031 is information on the position where light transmitted from the visible light communication light source on the liquid crystal panel 10321 is incident. Acquire communication light incident position information. Here, as an example, assuming that the number of pixels of the liquid crystal panel 10321 is 320 × 240 pixels, which is the same as the number of pixels of the captured image, the optical path length until the light passing through the second lens 201 reaches the second imaging element 1011, It is assumed that the light path length that the light passing through the two lenses 201 is reflected by the beam splitter 202 and reaches the liquid crystal panel 10321 is the same. Then, it is assumed that an image in the same shooting range as the second shot image shown in FIG. 8 is incident on the liquid crystal panel 10321. That is, the coordinate information of the liquid crystal panel 10321 and the coordinate information of the second captured image have a one-to-one correspondence. Therefore, the incident position information acquisition unit 1031 acquires the position information of the visible light communication light source region 52a as the communication light incident position information as it is. With such a configuration, the process of calculating the communication light incident position information from the position information of the visible light communication light source detected by the light source position detection unit 102 can be omitted, and the processing speed can be increased. Therefore, the communication light incident position information acquired here is the same as that shown in FIG. When the number of pixels of the second photographed image is different from the number of pixels of the liquid crystal panel 10321, the liquid crystal panel is in a position where each pixel in the second photographed image 1011 and the same image as the image indicated by this pixel are projected in advance. A correspondence table showing a correspondence relationship with one or more pixels 10321 is prepared in a storage medium (not shown), and communication light incident position information is acquired from position information of a visible light communication light source using the correspondence table. You may do it. Alternatively, coordinate conversion or the like may be performed.

  Next, the light shielding control unit 10322 controls the liquid crystal panel 10321 so that a region other than the position indicated by the acquired communication light incident position information among the pixels of the liquid crystal panel 10321 is a region that shields light. . Specifically, the liquid crystal panel 10321 is controlled such that only the pixel having the coordinates indicated by the communication light incident position information is a pixel that transmits light. As a result, as shown in FIG. 12, only the position indicated by the communication light incident position information 10321 of the liquid crystal panel becomes the light transmission region 1010. This region is a position where light emitted from the light source 51 that is a visible light communication light source is incident on the liquid crystal panel 10321.

  As a result, as shown in FIG. 13, the liquid crystal panel selectively receives only the light emitted from the light source 52 that is a visible light communication light source through the transmission region 1010 that is the region indicated by the communication light incident position information 10321. Is input. The light receiving means 1033 continuously outputs an electrical signal corresponding to the input light.

  Unlike the second image sensor 1011, the light receiving means 1033 such as a photodetector simply outputs an electrical signal corresponding to the intensity of light, and thus can perform high-speed processing. Therefore, information having a higher frequency than that of the second image sensor 1011 can be extracted from the electrical signal output by the light receiving unit 1033 according to the incident light.

  The information acquisition unit 1034 separates a signal having a frequency of “80 MHz” from the electrical signal output from the light receiving unit 1033. Then, the separated signals are sequentially demodulated, and information obtained by the demodulation is temporarily stored in a memory or the like (not shown).

  Next, it is assumed that the user performs an operation for releasing the shutter in the portable terminal 1 a in order to photograph the exhibit 71. For example, a shutter button (not shown) is pressed. In response to this operation, the first imaging unit 100 captures an image of the exhibit 71. An example of the first photographed image photographed by the first photographing unit 100 is shown in FIG. The first captured image is temporarily stored in a storage medium (not shown).

  The information acquisition unit 1034 acquires information transmitted by visible light communication in synchronization with imaging performed by the first imaging unit 100. Specifically, as described above, the information acquisition unit 1034 obtains one from the information temporarily stored in a memory or the like (not shown) by the information acquisition unit 1034 when the first imaging unit 100 performs imaging. A start code indicating the start point of the exhibit related information is detected. The information from the detected start code until the end code indicating the end point of the exhibit related information appears next is read from a memory (not shown) or sequentially received via the light receiving means 1033. The exhibit related information acquired here is exhibit related information regarding the exhibit 71 transmitted from the light source 52. An example of the exhibit related information acquired by the information acquisition unit 1034 is shown in FIG.

  The output unit 104 corresponds to the first shooting information obtained by shooting the exhibit 71 acquired by the first shooting unit 100 and the exhibit related information transmitted in synchronization with the shooting of the first shooting information by visible light communication. In addition, it is stored in a storage medium (not shown). Further, the first shooting information and the exhibit related information are combined and displayed on the display device 1051. A display example is shown in FIG.

  Next, it is assumed that the user moves in front of the exhibit 72 and points the first lens 200 toward the exhibit 72 in order to photograph the exhibit 72. In this case, the second photographing unit 101 acquires a second photographed image obtained by photographing the light sources 54 to 56 that are light sources located around the information of the exhibit 72. As a result, the portable terminal 1a exhibits the display of the exhibit 72 transmitted from the light source 55 that is a visible light communication light source when the first imaging unit 100 captures the exhibit 72 by the same processing as described above. Product related information will be acquired.

  Thus, when the user captures the exhibit 71, the mobile terminal 1a receives the exhibit related information regarding the exhibit 71 transmitted by visible light communication from the light source 51 that is a visible light communication light source. The first photographed image obtained by photographing the exhibit 71 and the exhibit related information received for the exhibit 71 by visible light communication can be acquired. When photographing the exhibit 72, the portable terminal 1a can receive exhibit related information about the exhibit 72 transmitted by visible light communication from the light source 55 which is a visible light communication light source. The first photographed image obtained by photographing the product 72 and the exhibit related information received for the exhibit 72 by visible light communication can be acquired. Thereby, the user can obtain detailed information about the subject to be photographed by the first photographing unit 100.

  The light source position detection unit 102 repeatedly performs the process of acquiring the position information of the visible light communication light source using the second captured images sequentially captured by the second imaging unit 101, and visible light communication with respect to the mobile terminal 1a. Even if the relative position of the light source changes, the position information of the visible light communication light source is newly acquired at the changed position. Then, communication light incident position information is acquired from the newly acquired position information, and the position of the transmission region of the liquid crystal panel 10321 is adjusted. As a result, it is possible to reliably receive the light output from the visible light communication light source. In particular, in this embodiment, since the position of the transmissive region is adjusted using the liquid crystal panel 10321, no physical operation is necessary, so that the position adjustment can be performed quickly. In addition, mechanical failure is unlikely to occur. Further, even if the shape of the light source is diversified, the transmissive region can be freely changed in accordance with the shape of the light source and the like, so that there is an advantage of high adaptability.

  As described above, according to the present embodiment, when a subject to be photographed is photographed by the first photographing unit 100, information on the subject to be photographed can be received by visible light communication, and thus the subject is photographed. The acquired image and the information regarding the subject to be photographed can be acquired simultaneously.

  Further, in the present embodiment, by transmitting the information about the subject to be imaged by visible light communication, the area for transmitting the information by adjusting the range to which the light from the light source is irradiated, It becomes possible to limit to a narrow area. As a result, it is possible to selectively transmit information even if the areas to be imaged and the objects to be imaged are close to each other. For example, when transmitting information about a subject to be imaged by a wireless communication technology such as so-called wireless LAN, it is very difficult to limit the transmission range of information to a narrow area. Specifically, as in the specific example described above, it is very difficult to transmit different information for each area to be photographed in the same room. On the other hand, in the present embodiment, since information about a subject to be imaged is transmitted by visible light communication, the communication range can be limited to a range in which light from a visible light communication light source can be received. As in the specific example described above, even in the same room, different information can be transmitted for each area to be imaged or for each object to be imaged.

  Further, according to the present embodiment, the position information of the visible light communication light source is acquired from the second photographed image, and the position information is the position where the light output from the visible light communication light source is incident using the position information. Communication light incident position information is acquired, and transmission target information that is information transmitted by visible light communication from the visible light communication light source at a position indicated by the communication light incident position information is acquired. For this reason, visible light communication can be performed even when other light sources exist, or when light output from light sources other than the photographed light source, such as ambient light or indirect illumination, is incident. It is possible to selectively receive only the light output from the detected light source that performs visible light communication, and to minimize the reception of light output from other light sources, and to perform visible light communication. Information transmitted from one light source to be performed can be received with high accuracy.

  In particular, the position of the visible light communication light source is detected using the second image sensor 1011 that can detect the light receiving position and the like, and the subject to be imaged transmitted from the visible light communication light source by visible light communication. Since information can be selectively received by the light receiving means 1033 such as a photodetector capable of high-speed response, it is possible to extract high-frequency information from information transmitted by being superimposed by visible light communication. . This makes it possible to transmit information modulated at a high frequency by visible light communication, and improve the data transfer rate.

  In this embodiment, a liquid crystal panel 10321 is particularly provided as means for selectively receiving light transmitted from the visible light communication light source. Then, a transmission region is formed in the liquid crystal panel 10321 in accordance with the position of the visible light communication light source, so that light transmitted from the visible light communication light source is selectively received. In such a configuration, the display image of the liquid crystal panel 10321 may be controlled so that the transmission region is formed at a position corresponding to the position where the visible light communication light source is detected in the captured image of the liquid crystal panel 10321. Therefore, calculations such as the direction of the visible light communication light source and the shape and size of the light from the light source are unnecessary. As a result, the processing speed can be increased. In addition, since a physical mechanism for selectively receiving light can be omitted, it is possible to make it difficult for a malfunction to occur due to wear or the like.

  In the present invention, the liquid crystal panel 10321 may be arranged at a position where light having passed through the second lens 201 is incident, and the arrangement is not limited. For example, as in the above embodiment, it may be disposed between the beam splitter 202 and the third lens 203, or may be disposed between the third lens 203 and the light receiving means 1033. Further, the liquid crystal panel 10321 may be disposed on the light receiving means 1033. Even when the liquid crystal panel 10321 is arranged at any position, similarly to the above embodiment, areas other than the incident position indicated by the incident position information acquired by the incident position information acquiring means 1031 by the light shielding control means 10322 block light. By controlling the liquid crystal panel 10321 so as to be in the region, the same effect as in the above embodiment can be obtained. In particular, since the liquid crystal panel 10321 may be disposed between the third lens 203 and the light receiving means 1033 to control transmission of light converged by the third lens 203, the size of the liquid crystal panel 10321 can be reduced. Thus, the cost can be reduced.

  In the following, the configuration and operation of an example of a transmission device that transmits information used for visible light communication in the above embodiment to a light source will be briefly described.

  FIG. 17 is a block diagram for explaining a configuration of a transmission device used for visible light communication.

  The transmission device 3 is connected to the light source 50 via a power communication line, a network, or the like. Here, the case where there is one light source 50 is taken as an example, but the light source 50 used for visible light communication may be plural, such as the light source 52 and the light source 55, as in the specific example described above. good.

  The transmission device 3 includes a transmission target information acquisition unit 301, a modulation unit 302, a marker signal output unit 303, a superimposition unit 304, and a transmission unit 305.

  The transmission target information acquisition unit 301 acquires transmission target information that is information to be transmitted by visible light communication. The transmission target information is information related to the photographing target of the photographing apparatus 1, such as the above-described exhibit related information. The acquisition described here is a concept including reading of transmission target information stored in a recording medium or the like, reception of transmission target information from the outside, and the like. Identification information for identifying a transmission destination light source may be added to transmission target information transmitted to a different light source. Alternatively, such information for identifying the transmission destination may be added when the modulation unit 302 described later performs modulation. The transmission target information acquisition unit 301 is realized by a communication unit, a reading unit from a recording medium, an MPU, a memory, or the like. The processing procedure of the transmission target information acquisition unit 301 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The modulation unit 302 modulates the transmission target information acquired by the transmission target information acquisition unit 301 with a predetermined frequency. The frequency to be modulated here is a frequency that can be processed by the light receiving means 1033 such as the above-described photodetector, and is preferably a frequency at which a sufficient data transfer rate can be obtained. For example, the frequency to be modulated is a frequency of several tens to 100 MHz.

  The marker signal output unit 303 outputs a marker signal that is information used to detect the position of the visible light communication light source. The marker signal is a signal having a frequency that can be received by the image sensor 1011 such as the image sensor described above. This frequency is, for example, about several tens KHz to several hundred KHz. The marker signal is information that changes the magnitude of the signal in, for example, a certain pattern. The marker signal output unit 303 may generate a marker signal having a predetermined frequency, or may read out a digital marker signal stored in advance in a storage medium (not shown), modulate it at a predetermined frequency, and output it. May be.

  The superimposing unit 304 superimposes one or more modulated transmission target information output from the modulation unit 302 and the marker signal output from the marker signal output unit 303. In addition, when the transmission apparatus 3 transmits these information by power line communication, you may superimpose such information on the electric power carrier wave. Note that the superimposing unit 304 may be omitted when the marker signal and the transmission target information are switched according to time or the like.

  The transmitting unit 305 transmits information including the transmission target information and the marker signal superimposed by the superimposing unit 304 to the light source 50 via a power communication line, a network, or the like. The transmission unit 305 is realized by communication means or the like.

  The light source 50 includes a light emitter (not shown) such as an LED, an incandescent lamp, or a fluorescent lamp. As the illumination light source 50 for performing optical wireless communication, for example, an LED (Light Emitting Diode) capable of blinking at high speed is preferably used. The luminous intensity of the light source 50 is adjusted by adjusting the voltage supplied to the light source 50. By changing the intensity of light emitted from the light source 50 according to the information transmitted from the transmission unit 305, for example, by blinking, the information transmitted from the transmission unit 305 can be transmitted by visible light communication. . The intensity of light generated by the light source 50 may be directly controlled by information transmitted from the transmission unit 305. Alternatively, the light source 50 receives a information (not shown) that receives information transmitted from the transmission unit 305, a control unit (not shown) that controls the emission intensity according to a signal received by the receiving unit, and the like. You may have. In addition, when the light source 50 receives information including identification information specifying the transmission destination from the transmission unit 305, information including identification information other than the identification information specifying the own device may be discarded. In this way, information can be selectively transmitted from the plurality of light sources 50.

  The transmission apparatus 3 modulates the transmission target information acquired by the transmission target information acquisition unit 301 by the modulation unit 302, and superimposes the modulated transmission target information and the marker signal output by the marker signal output unit 303. The information superimposed is transmitted from the transmission unit 305 to the light source 50. As a result, information obtained by superimposing the marker signal and the information to be transmitted can be transmitted from the light source 50 by visible light communication.

  In addition, each light source 50 is arrange | positioned in the vicinity of the position where what is considered to be a subject by the photographing apparatus 1 exists. The transmission device 3 transmits information related to what is considered to be a subject located in the vicinity of each light source 50. What is considered to be a subject is, for example, an exhibit of an art museum, a museum, or an exhibition hall as described above. It may also be a landscape such as a tourist attraction or a so-called shooting spot.

  In the present invention, an imaging system including such a transmission device 3 and the imaging device 1 may be configured.

  In the above-described embodiment, the case of receiving visible light communication has been described. However, the present invention can also be applied to optical wireless communication that can use light other than visible light, for example, infrared light and ultraviolet light. Even in such a case, the same effects as those of the above embodiment can be obtained. The optical wireless communication described here refers to communication using light waves having a very high frequency compared to radio waves, such as visible light, infrared light, and ultraviolet light. In this case, for example, an optical wireless communication light source is used instead of the visible light communication light source.

  Further, in each of the above embodiments, each process (each function) may be realized by centralized processing by a single device, or may be realized by distributed processing by a plurality of devices. Good.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, selected, generated, transmitted, and received by each component. In addition, information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing is retained temporarily or over a long period of time on a recording medium (not shown) even when not explicitly stated in the above description. It may be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  Note that the software that realizes the photographing apparatus in the above embodiment is the following program. That is, this program captures an image by capturing one or more light sources including a first image capturing unit that captures an image by capturing a subject and an optical wireless communication light source that is a light source that performs optical wireless communication. Light source position detection for detecting an optical wireless communication light source in a second captured image that is an image acquired by the second imaging unit and the second imaging unit, and acquiring positional information that is information on the position of the optical wireless communication light source And communication light incident position information, which is information on a position where light output from the optical wireless communication light source is incident, from the position information of the optical wireless communication light source acquired by the light source position detection unit, and the communication light An information acquisition unit that acquires transmission target information that is information transmitted by optical wireless communication from the optical wireless communication light source at an incident position indicated by incident position information, and a first captured image that is an image acquired by the first imaging unit When, Serial information acquiring unit is a program for functioning as an output unit for outputting the acquired transmission target information.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information or an output unit that outputs information is not included in the function realized by the program.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 18 is a schematic diagram illustrating an example of an external appearance of a computer that executes the program and realizes the imaging apparatus according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 18, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905, an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 19 is a diagram showing an internal configuration of the computer system 900. In FIG. 19, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program that causes the computer system 900 to execute the functions of the imaging apparatus according to the above-described embodiment is stored in the CD-ROM 921 or the FD 922, inserted into the CD-ROM drive 905 or the FD drive 906, and transferred to the hard disk 914. May be. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS) or a third-party program that causes the computer 901 to execute the functions of the imaging apparatus according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the photographing apparatus according to the present invention is suitable as a photographing apparatus for photographing a subject, and is particularly useful as a photographing apparatus that can obtain information describing a subject to be photographed. .

DESCRIPTION OF SYMBOLS 1 Shooting device 1a Portable terminal 3 Transmitting device 50, 51, 52, 53, 55 Light source 71, 72 Exhibit 100 First imaging unit 101 Second imaging unit 102 Light source position detection unit 103 Information acquisition unit 104 Output unit 200 First lens DESCRIPTION OF SYMBOLS 201 2nd lens 202 Beam splitter 203 3rd lens 301 Transmission object information acquisition part 302 Modulation part 303 Marker signal output part 304 Superimposition part 305 Transmission part 1001 1st image sensor 1002 1st image processing means 1011 2nd image sensor 1012 2nd Image processing means 1021 Light source detection means 1022 Light source selection means 1031 Incident position information acquisition means 1032 Selection transmission means 1033 Light reception means 1034 Information acquisition means 1051 Display device 10321 Liquid crystal panel 10322 Light shielding control means

Claims (8)

A first photographing unit for photographing an object and acquiring an image;
A second photographing unit that photographs one or more light sources including an optical wireless communication light source that is a light source that performs optical wireless communication, and acquires an image;
A light source position detection unit that detects an optical wireless communication light source in a second captured image that is an image acquired by the second imaging unit, and acquires position information that is information on a position of the optical wireless communication light source;
From the position information of the optical wireless communication light source acquired by the light source position detection unit, communication light incident position information, which is information on a position where light output from the optical wireless communication light source is incident, is acquired, and the communication light incident position information An information acquisition unit that acquires transmission target information that is information transmitted by optical wireless communication from the optical wireless communication light source at the incident position indicated by:
An imaging apparatus comprising: a first captured image that is an image acquired by the first imaging unit; and an output unit that outputs transmission target information acquired by the information acquisition unit. The information acquisition unit acquires information transmitted by optical wireless communication in synchronization with imaging performed by the first imaging unit,
The output unit associates and outputs a first captured image that is an image acquired by the first imaging unit and information acquired by the information acquisition unit in synchronization with the imaging of the first captured image. 1. The photographing apparatus according to 1. The information acquisition unit recognizes a start point and an end point of optical wireless communication information transmitted by optical wireless communication at a time when photographing is performed by the first photographing unit, and optical wireless in a range from the recognized start point to the end point The imaging apparatus according to claim 2, wherein the communication information is acquired as information transmitted in synchronization with imaging. The imaging apparatus according to any one of claims 1 to 3, wherein the transmission target information is information relating to a subject to be imaged by the first imaging unit. The portable terminal provided with the imaging device in any one of Claims 1-4. A transmission target information acquisition unit that acquires transmission target information that is information received by the imaging apparatus according to claim 1 and that is information to be transmitted by optical wireless communication;
A modulation unit that modulates the transmission target information acquired by the transmission target information acquisition unit;
A marker signal output unit that is information used to detect a position of a light source that performs optical wireless communication, and outputs a marker signal having a frequency different from a frequency at which the modulation unit performs modulation.
A superimposing unit that superimposes transmission target information output from the modulation unit and a marker signal output from the marker signal output unit;
A transmission apparatus comprising: a transmission unit that transmits information including transmission target information and a marker signal superimposed by the superimposition unit. An imaging method performed using a first imaging unit, a second imaging unit, a light source position detection unit, an information acquisition unit, and an output unit,
A first imaging step in which the first imaging unit acquires an image by imaging a subject;
A second imaging step in which the second imaging unit captures an image by capturing one or more light sources including an optical wireless communication light source that is a light source for performing optical wireless communication;
The light source position detection unit detects the optical wireless communication light source in the second captured image that is the image acquired in the second imaging step, and acquires the position information that is the position information of the optical wireless communication light source A detection step;
The information acquisition unit acquires communication light incident position information, which is information of a position where light output from the optical wireless communication light source is incident, from the position information of the optical wireless communication light source acquired in the light source position detection step, An information acquisition step of acquiring transmission target information that is information transmitted by optical wireless communication from the optical wireless communication light source at the incident position indicated by the communication light incident position information;
An imaging method comprising: an output step in which the output unit outputs a first captured image that is an image acquired in the first imaging step, and transmission target information acquired by the information acquisition unit. Computer
A first photographing unit for photographing an object and acquiring an image;
A second photographing unit that photographs one or more light sources including an optical wireless communication light source that is a light source that performs optical wireless communication, and acquires an image;
A light source position detection unit that detects an optical wireless communication light source in a second captured image that is an image acquired by the second imaging unit, and acquires position information that is information on a position of the optical wireless communication light source;
From the position information of the optical wireless communication light source acquired by the light source position detection unit, communication light incident position information, which is information on a position where light output from the optical wireless communication light source is incident, is acquired, and the communication light incident position information An information acquisition unit that acquires transmission target information that is information transmitted by optical wireless communication from the optical wireless communication light source at the incident position indicated by:
The program for functioning as an output part which outputs the 1st picked-up image which is an image which said 1st imaging | photography part acquired, and the transmission object information which the said information acquisition part acquired.

Images