JP4210955B2 - Imaging apparatus, imaging control method, and imaging control program - Google Patents

Description

  The present invention relates to an imaging apparatus, an imaging control method, and an imaging control program. More specifically, the imaging apparatus, an imaging control method, and an imaging control that measure various distances to an imaging target and perform various controls using the measurement results. Regarding the program.

  2. Description of the Related Art Conventionally, there are imaging apparatuses such as digital cameras that have an AF (automatic focusing) function. As a method for measuring the distance to a subject in the AF function, there are generally a PSD type mounted on a compact camera or the like, and a TTL contrast type (or precision focus type) mounted on an interchangeable lens single-lens reflex camera or the like. Are known.

  Here, the PSD type AF technology will be described. The PSD type emits infrared light from an infrared light emitting unit provided on the front surface of the camera, and reflects the reflected light that is irradiated and reflected on the subject. By receiving light with a distance sensor provided in, the distance sensor obtains the distance to the subject from the angle between the infrared light and the reflected light and performs AF processing.

  Further, as a distance measuring technique with higher accuracy than the PSD type, some optical measuring instruments use a laser as described in Patent Document 1, for example. In the technology using this laser beam, the laser beam emitted from the built-in semiconductor laser 20 receives the reference beam received through the beam splitter and the measurement beam reflected by the subject by the light receiving element 29, The distance to the subject is measured by the time difference (optical frequency difference) between the two lights, and focus detection is performed. According to the distance measurement using such a laser beam, the focal length can be obtained with high accuracy with less dependence on the distance as compared with the PSD type depending on the surrounding environment and the wavelength region to be used.

Japanese Patent Laid-Open No. 10-90591 (pages 3 to 4, FIG. 1 and FIG. 2)

  However, the distance measurement technique using the laser beam described above is not adopted for consumer devices due to safety issues, and when the distance to the subject is relatively far, the distance to the subject is accurately determined. In addition, it is impossible to measure safely and reflect the measurement results in various photographing controls.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus, an imaging control method, and an imaging control program that can accurately measure the distance to a subject and perform various imaging controls using the measurement results. There is to do.

The invention according to claim 1 is detected by the imaging means, the area detecting means for detecting an area having a predetermined luminance from the imaging surface imaged by continuously driving the imaging means, and the area detecting means. Information acquisition means for acquiring information related to the size of the object corresponding to the area from the temporal luminance change in the area, the area of the area on the imaging surface, the imaging distance, and the information acquisition means A distance calculating unit that calculates a distance between the device and the object based on the information acquired by the control unit, and a control unit that controls each process related to the imaging operation based on the distance calculated by the distance calculating unit. It is characterized by comprising.
According to a second aspect of the present invention, in the first aspect of the present invention, the control unit is configured to control the processing between the apparatus and the object calculated by the distance calculation unit as control of each process related to the imaging operation. Based on the distance, reproduction or storage of information acquired by the information acquisition unit is controlled.
The invention according to claim 3 further includes a display means in addition to the invention according to claim 1 or 2, wherein the control means is calculated by the distance calculation means as control of each process related to the imaging operation. Control is performed so that auxiliary information related to imaging is displayed on the display unit based on the distance between the device and the object.
The invention according to claim 4 further includes a focusing unit in addition to the invention according to claim 1, and the control unit is calculated by the distance calculation unit as control of each process related to the imaging operation. The focusing means is controlled based on the distance between the device and the object.
According to a fifth aspect of the present invention, in the first aspect of the present invention, the control unit is configured to control the processing between the apparatus and the object calculated by the distance calculation unit as control of each process related to the imaging operation. Control is performed so as to process an image picked up by the image pickup means based on the distance.
According to a sixth aspect of the present invention, in the first aspect of the present invention, the control unit is configured to control the processing between the apparatus and the object calculated by the distance calculation unit as control of each process related to the imaging operation. Based on the distance, the optical system mechanism included in the apparatus is controlled.
According to a seventh aspect of the present invention, there is provided an area detection step for detecting an area having a predetermined luminance from an imaging surface imaged by continuously driving the imaging section, and luminance in the area detected by the area detection step. Information acquisition step for acquiring information on the size of the object corresponding to the region, the area of the region on the imaging surface, the imaging distance, and the information acquisition step. A distance calculating step for calculating a distance between the device and the object based on the information obtained, and a control step for controlling each process related to the imaging operation based on the distance calculated in the distance calculating step. It is characterized by becoming.
The invention according to claim 8 is an area detecting means for detecting an area having a predetermined luminance from the imaging surface imaged by continuously driving the imaging section of the computer, and in the area detected by the area detecting means. Information acquisition means for acquiring information related to the size of the object corresponding to the region, the area of the region on the imaging surface, the imaging distance, and the information acquisition means obtained from a temporal change in luminance. Based on the information, the distance calculation means for calculating the distance between the device and the object, and the control means for controlling each process related to the imaging operation based on the distance calculated by the distance calculation means. Features.

  According to the present invention, an area having a predetermined luminance is detected from an imaging surface that has been continuously imaged, and an object corresponding to the area having the luminance is detected from a temporal change in luminance in the detected area. While acquiring information on the size, the area of this region on the imaging surface, the imaging distance, and the distance between the object and the object are calculated based on the acquired information, and based on the calculated distance Each process related to the imaging operation is controlled. Therefore, it is possible to accurately measure the distance to the subject and perform various shooting controls using the measurement result.

  Hereinafter, application to a digital camera will be described as an example with reference to the drawings.

<Principle explanation>
FIG. 1 is a configuration diagram of a digital camera showing an example of an application system of the present embodiment. In this figure, a digital camera 10 includes a central control unit 14 that includes a CPU 11, a ROM 12, a RAM 13, various peripheral circuits (not shown) and the like, and typically a microprocessor formed on a single chip, and the periphery of the central control unit 14. Are provided with at least the following units necessary for the operation of the digital camera 10.

  The imaging unit 15 includes an optical system 16 including a photographing lens, a diaphragm mechanism, a focusing mechanism, a zoom mechanism, and the like, and an imaging device 17 including a two-dimensional image sensor such as a CCD sensor or a CMOS sensor. The operation of the imaging unit 21 (aperture size and zoom magnification, that is, adjustment and focusing of the shooting angle of view α, and exposure and readout operations of the imaging device 17) is performed by autofocus control including a central control unit 14 and a step motor 181. Control is performed by the control from the imaging control unit 19 that has received an imaging operation instruction from the unit 18. The photographing operation instruction from the central control unit 14 is a frame image reading operation instruction at a predetermined frame rate (for example, several tens to several hundreds frame rate per second) for photographing composition confirmation (so-called through image) or moving image photographing. , High-resolution still image shooting operation instructions, prior operation instructions such as aperture value setting and zoom magnification setting necessary for these operations, and the shooting operation instruction from the autofocus control unit 18 is an optical system 16 focusing (focusing) operation instructions.

  In response to a shooting operation instruction from the central control unit 14, a frame image for shooting composition confirmation or a moving image is periodically read out from the imaging unit 21 at the above frame rate, or a high-resolution still image A frame image is read out. These frame images are converted into digital signals in the image processing unit 21 and subjected to necessary image processing (for example, gamma correction processing), and then taken into the central control unit 14 via the FIFO buffer 22. It is.

  The operation unit 23 includes various operations necessary for an input interface for user operation of the digital camera 10, such as a power switch, a mode switch between image capturing and image reproduction, a shutter button for performing still image capturing and moving image capturing, A menu button for displaying a setting menu, a selection button for selecting the menu item, and selecting an image desired to be reproduced in the image reproduction mode are included.

  The display control unit 24 converts various display data appropriately output from the central control unit 14, for example, through-image display data, menu screen display data, and image playback screen display data into a predetermined display format. Then, the data is output to the display unit 25 configured by a flat display device such as a liquid crystal display. The display unit 25 is provided with a touch panel 26, and the touch detection unit 27 detects touch coordinates such as a finger or a pen on the touch panel 26 and outputs the detection result to the central control unit 14.

  The image storage unit 28 is configured by a nonvolatile mass storage device such as a flash memory, a hard disk, or an optical disk (the stored contents are not lost even when the power is turned off). Used to store and store the generated images. Each image stored and stored is, for example, a compressed file in JPEG format or the like, or an uncompressed raw data file (so-called RAW file), and the storage area of these images is directly under the root in the file system. Alternatively, it may be a folder having a single hierarchy or a multi-hierarchy structure appropriately created immediately below the root. In addition, the image storage unit 28 may be a fixed type, or may be a general-purpose memory device that can be detached from the digital camera 10 and attached to a personal computer (not shown), for example.

  The external interface 29 is, for example, a data input / output unit compatible with a general-purpose protocol such as USB or IEEE 1394. When the external interface 29 is necessary, a captured image can be exchanged with a personal computer (not shown) as necessary. Transfer (transfer images stored and stored in the image storage unit 28 to a personal computer) and read (read images from the personal computer to the image storage unit 28) can be performed.

  The power supply unit 30 includes a rechargeable secondary battery or a disposable primary battery, and supplies a power supply voltage necessary for the operation of each unit of the digital camera 10 including the central control unit 14.

  The azimuth sensor 31 and the elevation sensor 32 both detect the photographing direction of the digital camera 10 (the direction of the optical axis of the optical system 16). For example, the azimuth sensor 31 has a azimuth with magnetic north as 0 degrees. Then, the elevation sensor 32 detects an elevation angle (or depression angle) with the horizontal as 0 degree.

  Here, the illustrated subject 20 simulates a person, but this is an example. The important point is that a light emitter 33 is provided at the position of the subject 20 as an object for transmitting arbitrary data (details will be described later) subjected to luminance modulation by light. Although details will be described later, the digital camera 10 receives the image including the illuminant 33 in time series with the imaging device 17 (sequentially captures), and acquires the measurement target included in the image. As described above, the data included in the luminance modulation region in the light emitter 33 can be restored, and the distance D to the light emitter 33 can be measured and acquired based on the received image and the restored data. .

  FIG. 2 is a configuration diagram of the light emitter 33. In this figure, a light emitter 33 modulates the light source 34 that emits light in the visible light region, the data memory 35 that stores data to be transmitted, and the data stored in the data memory 35, and the light source 34 with the modulation information. And a light emission window 37 having a predetermined shape and a predetermined size. Further, the data memory 35 includes a guide data memory 351 for holding arbitrary information, data on the shape of the light emission window 37 (here, “circular”) and size data of the light emission window 37 (here, the diameter is “R” because it is circular). ) And a self-size data memory 352 for holding position (latitude / longitude and altitude) data of the light emitter 33.

  The light emission control unit 36 also includes two types of preamble data, which are detection / capture preamble data (for measurement) and detection / capture preamble data (for data body), which will be described later, and two different brightness change patterns ( Hereinafter, a pattern data memory 361 that holds the first pattern series SA and the second pattern series SB), a timing generator 362, and a control unit 363 are provided.

  The timing generator 362 generates a stable clock signal having a predetermined period. This clock signal is synchronized with the clock signal of the imaging device 17 of the digital camera 10.

  The control unit 363 sequentially extracts bit data from the data stored in the pattern data memory 361, the guide data memory 351, and the self-size data memory 352 in synchronization with the clock signal from the timing generator 362, and the bit The value (“1” data or “0” data) is determined, and if it is “1” data, the first pattern series SA is extracted from the pattern data memory 361, while if it is “0” data, the pattern data memory 361. The operation of taking out the second pattern series SB and outputting the first pattern series SA or the second pattern series SB to the light source 34 is repeated for the number of bits of data to be transmitted.

  The light source 34 performs a blinking operation of emitting light at a timing corresponding to “1” in the first pattern sequence SA and the second pattern sequence SB, and turning off (or reducing luminance) at a timing corresponding to “0”. Thus, the light P having a luminance change in time series is output through the light emission window 37.

  In the figure, the light emitter 33 is held by a person as the subject 20, but for example, a fixed structure such as a signboard or a guide plate is used as the subject 20, and the fixed structure such as a signboard or a guide plate is used. You may make it provide the light-emitting body 33 for every. In this case, the data stored in the data memory 35 of each light emitter 33 is downloaded sequentially from the installed building or a server provided outside the building via a network such as a LAN. It may be. In addition, the light source 34 blinks with the pattern series “1” and “0”. However, when the pattern series covers multi-value data, not only “lighting” and “lighting off” but also multiple stages. You may make it light-emit with the brightness | luminance of.

  As described above, the data memory 35 stores at least shape data indicating the shape of the light emitter 33 and size data of the light emitter 33 as data to be transmitted. For example, referring to FIG. 2, the data memory 35 stores shape data indicating “circular” as the shape of the light emission window 37 and “R” indicating the diameter as the size thereof as size data. The light emission controller 36 modulates these data. Also, regarding the modulation method of the shape data and size data to be transmitted performed by the light emission control unit 36, for example, the above shape data and size data are binary digital data composed of logic 0 and logic 1, and “0 "Data is assigned a luminance change pattern (first pattern series SA) with a corresponding time series, and" 1 "data is assigned a luminance change pattern with a time series different from the" 0 "data. It is desirable to assign (second pattern series SB). It is desirable that these two luminance change patterns change in the same cycle and change in a cycle different from the cycle existing in the natural world, such as a cycle standardized by a commercial power supply or disturbance light.

  On the other hand, the digital camera 10 has the above-described configuration (see FIG. 1), and operates as a general digital camera, that is, takes a still image or a moving image and stores the image file as an image storage unit 28. The “shooting function” stored and stored in the image storage unit and the “playback function” that reads out any image file stored and stored in the image storage unit 28 and reproduces and displays it on the display unit 25 can be appropriately executed. In addition, the “ranging function” specific to the present embodiment, that is, by receiving and acquiring an image including the light emitting body 33 at the position of the subject 20 in time series, light emission is performed as a measurement target included in the image. The function of restoring the data included in the luminance modulation area of the body 33 and measuring and acquiring the distance D to the light emitter 33 based on the received image and the restored data is implemented. It has become possible way.

  This “ranging function” is mainly provided by the function of the central control unit 14 of the digital camera 10. That is, the central control unit 14 of the digital camera 10 controls the operation of each unit of the digital camera 10, and in particular in the present embodiment, it is stored in the capture cycle control of the imaging device 17 and the FIFO buffer 22. The distance D to the light emitter 33 based on the data reading and the size of the image of the light emitter 33 formed on the light receiving surface 17a of the imaging device 17 and the shape data and size data restored by the signal restoration means 14c described later. Various processes using the measurement and the measurement result are controlled.

  FIG. 3 is a conceptual diagram showing a part of the storage space of the RAM 13 of the central control unit 14 of the digital camera 10 and some functional blocks realized by the central control unit 14. That is, (a) is a part of the storage space of the RAM 13 of the central control unit 14 of the digital camera 10, and (b) is some functional blocks realized by the central control unit 14. In these figures, the storage space of the RAM 13 includes areas such as an imaging distance data storage unit 13a, an imaging distortion correction data storage unit 13b, a distance calculation data table storage unit 13c, a detection data list storage unit 13e, and the like. The control unit 14 includes functions such as a pattern data memory 14a, a signal area detection unit 14b, a signal restoration unit 14c, and a work memory 14d.

  The photographing lens included in the optical system 16 is composed of, for example, a single convex lens, and connects the angle of view α including the light emitter 33 around the optical axis A (see FIG. 2) to the imaging device 17 at the subsequent stage. Provided to image. The imaging device 17 is composed of an image sensor such as a CCD or CMOS in which a plurality of imaging elements are regularly arranged, and the light emission mode of the light emitter 33 captured in two dimensions is electrically converted into an image based on the light reception ratio on the light receiving surface 17a. The device converts the signal into a signal and outputs the signal, and outputs the signal at a predetermined frame rate (for example, 30 FPS) based on the control of the central control unit 14. The device used as the imaging device 17 is not limited to this as long as the light emission mode of the light emitter 33 can be acquired in two dimensions. For example, a configuration in which a plurality of light receiving elements, for example, photodiodes are arranged. It may be. Further, since the imaging device 17 is composed of an image sensor such as a CCD or CMOS, the reference numeral 17a is preferably referred to as an imaging surface, but in the description of the principle, it will be described as a light receiving surface.

  The ROM 12 stores various control programs executed by the CPU 11 of the central control unit 14, and the RAM 13 is used as an execution area for these control programs, and also stores various data shown in FIG. An imaging distance data storage unit 13a for storing an imaging distance d between the light receiving surface 17a of the device 17 and a photographic lens included in the optical system 16 is provided. In the present embodiment, the photographic lens included in the optical system 16 has a single convex lens fixedly installed. However, in an imaging apparatus equipped with an optical zoom, there are cases where the imaging distance varies depending on the movement of the lens. In this case, it is desirable to use the imaging distance or focal length obtained by adjusting the position of the lens.

  Under the control of the central control unit 14, the signal restoration unit 14c sequentially captures the light emission modes of the light emitters 33 that are output in time series by being continuously imaged by the imaging device 17 at a period of 30 FPS. Based on the light emission mode of the illuminant 33 acquired in the above, the luminance-modulated data is restored to the data stored in the data memory 35. For example, in the case of modulation by the above modulation method, the data is demodulated by a reverse method to restore the shape data and size data.

  Similar to the pattern data memory 361 of the light emission control unit 36, the pattern data memory 14a includes two types of preamble data, that is, detection / capture preamble data (for measurement) and detection / capture preamble data (for data body), and Two different types of luminance change patterns (first pattern series SA and second pattern series SB) are held.

  When a pixel whose luminance changes in time series is detected from a plurality of frames of image signals held in the FIFO buffer 22, the signal area detection unit 14b detects from the pixel group whose luminance changes at the same timing as this luminance change. A function of specifying a pixel region. The signal restoration unit 14 c is a data format 38 that is sequentially buffered in the FIFO buffer 22 when pixels whose luminance changes in time series are detected from the image signals of a plurality of frames held in the FIFO buffer 22. Output bit data of “1” and “0” corresponding to the detected luminance change of the pixel from the frame data corresponding to the bit length of the first pattern sequence SA, and It is determined whether it matches any of the second pattern series SB, and if it matches any of them, a bit corresponding to this pattern is output, and size data, position data, and guide data are output from the output bit. Process to restore to. The work memory 14d holds an image on the imaging surface (light receiving surface 17a) of the specified pixel region.

  The central control unit 14 includes a RAM 13 for temporarily storing each data obtained by the processing described later, takes in the data restored by the signal restoration unit 14c, and is set to the measurement mode based on the data. If so, the distance measurement to the light emitter 33 and the current position measurement of the digital camera 10 are executed, and the measurement results are output to the display unit 25. On the other hand, if the guide mode is set, the light reception (imaging) is performed. ), And the guide data stored in the data record of the detection data list storage unit 13e is output to the display unit 25. The imaging distance data storage unit 13a stores the imaging distance d in the above description of the principle. The imaging distortion correction data storage unit 13b stores data for correcting distortion due to the characteristics of the photographing lens of the optical system 16 for the image formed on the imaging device 17.

  The distance calculation data table storage unit 13c stores formulas (1) and (2) in the principle description to be described later. The current position data storage unit 13d is for holding its own position information obtained by the central control unit 14, and the detection data list storage unit 13e is up to the light emitter 33 that has been arithmetically processed by the central control unit 14. Distance D, self-position (coordinates and altitude), and guide data acquired by receiving light.

  The detection data list storage unit 13e detects the distance and position (coordinates and altitude) to the light emitter 33 when the light emitter 33 is detected from the imaging surface (light receiving surface 17a) output in time series from the imaging device 17. In the present embodiment, the data is stored in the form of data records. This is because, for example, when a plurality of light emitters are detected from the imaging surface (light receiving surface 17a), each light emitter is stored individually.

  Here, the principle of distance calculation will be described. First, as described above, R is obtained as size data by receiving luminance-modulated data, and the diameter of the light emitter 33, d is an imaging distance, and is the maximum of the range that can be received by the photographing lens included in the optical system 16. The angle is α.

  FIG. 4 is a diagram showing an image formed on the light receiving surface 17a of the imaging device 17, and as shown in this figure, the horizontal length of the light receiving surface 17a is H, the vertical length is V, and the light emitter. The image 33 is denoted by 33a, and the diameter of the image 33a is denoted by r.

  Then, the angle β can be obtained from the diameter r of the image 33a and the imaging distance d.

FIG. 5 is a conceptual diagram of calculation of the distance D. As shown in this figure, the imaging position of the photographic lens included in the optical system 16 is used as a reference, and the triangle formed by the angle β, the distance D, and R / 2, the angle β, the distance d, and r. It can be seen that the triangle formed by / 2 is similar. Therefore, the distance D can be measured by using a trigonometric function expression from these relationships. The method of calculating the distance D is as shown in the following equations (1) and (2).
D = (R / 2) / {(tan (β / 2)) (1)
β = r / 2d (2)

  FIG. 6 is a diagram illustrating an example of a data format 38 output from the control unit 363 to the light source 34 and transmitted as the light P. The data format 38 includes a detection / capture preamble data portion (for measurement) 38a, a size data portion 38b, a position data portion 38c, a detection / capture preamble data portion (for guide data) 38d, and a guide data portion 38e. These are output cyclically with 1 as a unit.

  The data stored in the detection / capture preamble data portion (for measurement) 38a is data detected when the digital camera 10 sets the measurement mode when the data format 38 is received by the digital camera 10. By receiving this data portion, a required processing operation is executed for the calculation related to the distance and position to the light emitter 33.

  The data stored in the size data portion 38 b is the shape data (here “circular”) of the light emission window 37 and the size data “R” of the light emission window 37 among the data stored in the self size data memory 352. The digital camera 10 is for measuring the distance to the light emitter 33 based on these data.

  The data stored in the position data unit 38c is the position (latitude / longitude and altitude) data of the light emitter 33 among the data held in the self-size data memory 352, and the digital camera 10 is based on this data. This is for measuring the direction and the self-position of the light emitter 33 viewed from the digital camera 10.

  The data stored in the detection / capture preamble data portion (for guide data) 38d is data detected when the digital camera 10 sets the guide mode when the digital camera 10 receives the data format 38. Yes, by receiving the data portion, the processing operation for restoring and reproducing and outputting the data set in the guide data portion 38e is executed.

  The data stored in the guide data section 38e is data stored in the guide data memory 351. In the digital camera 10, based on this data, for example, options such as route guidance, sightseeing guidance, and auxiliary information related to imaging are provided. This is for causing the processing to be executed as necessary.

  FIG. 7 is a diagram showing a flowchart in the present embodiment. This flowchart is roughly composed of a signal area detection processing block S1, a signal restoration processing block S2, and a measurement processing block S3.

  First, in the signal area detection processing block S1, an image formed on the imaging surface (light receiving surface 17a) of the imaging device 17 is used as frame data, and first, the number of frames corresponding to the number of bits of the preamble pattern is sequentially input to the FIFO buffer 22. Buffering is performed (step S11). Then, in the plurality of buffered frame data, it is determined whether or not there is a pixel whose luminance has changed. Specifically, in a plurality of buffered frame data, the maximum luminance exceeds a predetermined peak, and it is determined whether or not there is a periodically changing pixel, so that the time series from the pixel is determined. In step S12, it is determined whether or not luminance-modulated data is transmitted.

  If a pixel whose luminance has changed cannot be detected, the process of step S14 described later is performed, and the process returns to step S11. However, if a pixel is detected, preamble pattern data (for measurement) and preamble pattern data are detected from the pattern data memory 14a. (For guide data) are read out, and these preamble patterns are collated with the time-series luminance changes from the detected pixels (step S13). As a result of the collation, if it does not completely match any preamble pattern data, the frame data buffered in the FIFO buffer 22 is discarded, assuming that data cannot be obtained from the currently detected pixel (step S14). Then, the process returns to step S11 again, but if it matches any preamble pattern data (including partial match), the azimuth sensor 27 is driven to acquire the imaging direction (step S15), and the elevation sensor 28 is driven to obtain an imaging elevation angle (horizontal angle) γ (step S16).

  In step S13, if any of the preamble pattern data matches, it is determined that the pixel detected this time is a pixel transmitting data, and the luminance of the signal region detection unit 14b is changed at the same timing. Control is performed so as to specify a pixel region composed of a certain pixel group (step S17). The imaging direction acquired in step S15 and the imaging elevation angle acquired in step S16 are temporarily held in the RAM 13 in the central control unit 14, and the specified pixel area is stored in the work memory 14d of the signal area detection unit 14b. The image is stored (step S18).

  Next, the signal restoration processing block S2 will be described. In the signal restoration process, first, frame data corresponding to the number of bits of the data format 38 is sequentially obtained from the pixel area specified in step S17 of the signal area detection processing block S1 (step S21), and stored in the FIFO buffer 22. On the other hand, with respect to the signal restoration means 14c, the first pattern series is processed with respect to the process of converting the luminance changing region into bit data of “1” and “0” and the bit data obtained through this process. SA, collation with the second pattern series SB, bit output processing, and processing for restoring the size data, position data, and guide data from the output bits are performed (step S22). Of these restored data, the size data and position data are temporarily stored in the RAM 13 of the central control unit 14, and the guide data are stored in the data record of the detection data list storage unit 13e (step S23).

Next, the measurement processing block S3 will be described in detail.
FIG. 8 is a diagram showing an operation flowchart of the measurement processing block S3.

  In the measurement processing block S3, first, the image of the pixel area stored in the work memory 14d is read, the shape of the light emitter 33 is specified for this pixel area, and the measurement axis is set (step S31). Next, each pixel on the measurement axis is weighted to determine the area of the “image” of the light emitter included in the pixel region (step S32).

Specifically, a method for determining the measurement axis, weighting, and area of the “image” will be described.
FIG. 9 is a specific explanatory diagram of a method for determining the measurement axis, weighting, and area of “image”. 9A, in the frame data sequentially obtained from the imaging surface (light receiving surface 17a), the maximum luminance exceeds a predetermined peak, and the pixel region of the illuminant 33 is periodically changed. Assume that the image 33a is set as a candidate. In this figure, the image 33a is an ellipse because it is assumed that the light-emitting body 33 is looked up (or looked down) from an oblique 45 degree direction. Of course, when the illuminant 33 is viewed from the front, it becomes a circular image 33a.

  Referring to the enlarged view of FIG. 9B, the central control unit 14 determines the dot of which the maximum luminance exceeds a predetermined peak for the image 33a having the height 277 (9 dots) and the width 223 (5 dots). The longest column including many pixels having the maximum luminance peak in one row or column is determined as the measurement axis. Then, weighting corresponding to the luminance is performed on the peripheral pixels around the measurement axis. For example, considering FIG. 9B, the maximum weight value “1” is set for the pixel range 224 having the highest luminance, the weight value “0.6” is set for the peripheral range 225, and The weight value “0.3” is set for the outermost edge range 226, and the height 277 (9 dots) is set as the measurement axis. Then, by calculating “20” for the pixel range 224, 7.2 for the pixel range 225, and 2.4 for the pixel range 226 from these weightings, the area of the “image” 33a is 29.6.

  Note that the measurement axis, the weighting determination method, and the area calculation method are not limited thereto, and other methods can be used as long as there is a method that can determine the area more accurately.

  As described above, when the area of the “image” of the light emitter is determined, the distortion correction data based on the characteristics of the photographing lens included in the optical system 16 is read from the imaging distortion correction data storage unit 13b for the shape of the image. The distortion is corrected using the data (step S33), the imaging distance d is read from the imaging distance data storage unit 13a, and β is calculated from the measurement axis and the imaging distance d (step S34).

When β is calculated in step S35, next, using the β, the previous equation (1) is read from the distance calculation data table storage unit 13c, and the size data stored in the RAM 13 of the central control unit 14 is used to calculate the value in FIG. The distance D is calculated (step S35). Since the shape of the “image” 33a is set as an “elliptical shape” when the circle is viewed from about 45 °, the size data acquired as “a circle with a diameter“ R ”” in step S24. Is corrected from the set shape as “an ellipse having a diagonal √2L”. Then, a distance D ′ (horizontal distance) to the position immediately below the light emitter 33 is calculated from the distance D and the shooting elevation angle γ obtained in step S31 by the following equation (3) (step S36).
D ′ = Dcosγ (3)

  Further, the position of the digital camera 10 is calculated from the distance D ′ calculated as described above, the shooting direction of the digital camera 10 acquired from the orientation sensor 27, and the position data acquired in step S24 (step S37). Distance data D ′ and position data are stored in the data record of the detection data list storage unit 13e (step S38).

The data registered in this way can be used in the digital camera 10 as follows.
FIG. 10 is a conceptual diagram showing an example of use of registration data. In this figure, (a) is a display example when the digital camera 10 is set to the measurement mode, and (b) is a display example when the digital camera 10 is set to the guide mode. In (a), on the screen of the display unit 25, the distance information to the light emitter 33 (for example, “distance to the target 3 m”) and the current position information of the digital camera 10 (for example, “your position north latitude 35” “4625.75 east longitude 139 ° 1843.69 ″) is displayed surrounded by a balloon figure at the screen corner. Also, in (b), on the screen of the display unit 25, predetermined guidance information transmitted from the light emitter 33 (for example, “150m ahead” is surrounded by a balloon figure at the screen corner). ing.

  As described above, according to the combination of the digital camera 10 and the light emitter 33 of the present embodiment, the distance measurement display, the current position display, the route guidance, and the like are performed by the information transmission by the luminance modulated light from the light emitter 33. Can do. In addition, the distance measuring technique based on such a principle description can control each process related to the imaging operation in the application examples listed below.

<Specific application example 1>
FIG. 11 is a conceptual diagram showing an application example of a system including an advertisement exhibit and an information terminal having a camera function. In the figure, two persons A and B are shown. Both of these persons I and B possess the information terminal (digital camera in the explanation of the principle) 10, both of which are information terminals 10 having a camera function toward an advertising exhibit 39 installed at a predetermined place. Is holding. Here, when the distance from the advertisement exhibit 39 to the person A is D1, and the distance from the advertisement exhibit 39 to the person B is D2, D1 <D2. In other words, the character A is located close enough to visually read the characters on the advertisement exhibit 39, and the character B is located far enough to prevent the characters on the advertisement exhibit 39 from being visually read. Here, only two persons I and B are shown, but these persons I and B are representatives of people who are located close enough to read the characters of the advertisement exhibit 39 visually. The character B is a representative of people who are located so far that the characters of the advertisement exhibit 39 cannot be visually read.

  In the advertisement exhibit 39, predetermined advertisement information is posted. In the case of the illustrated example, this advertisement information is a printed image of a product (personal computer) and an explanation thereof, and further includes a note “Coupon is being distributed!”.

  This advertisement exhibit 39 is irradiated with visible illumination light 40a from the lamp 40, and this irradiation range corresponds to a light source for visible light communication similar to the light emitter 33 described above. Hereinafter, this range is referred to as a light emitter 40b for the sake of convenience in order to maintain consistency with the above description.

  The main role of the lamp 40 is to brightly illuminate the advertising exhibit 39 with the illumination light 40a so that it can be seen by more people. As a subordinate role, the illumination light 40a is applied to the human eyes. By changing the luminance at such a short time interval that cannot be identified, the desired data is transmitted by visible light communication according to the luminance change pattern of the light emitter 40b (light due to the illumination light 40a being reflected).

  In the display unit 25 of the information terminal 10 of the person A located near the advertisement exhibit 39, an information frame is displayed together with a captured image 42 in which the advertisement exhibit is captured as shown by reference numeral 41 in the figure. 43 is displayed, and ““ Attached information (discount coupon) was received and saved inside this information frame 43. "" Is displayed. In addition, although it is set as the discount coupon here, this is only an example. It may be information for sales promotion, for example, link information (URL information) on the Internet for providing a special service.

  On the other hand, the display unit 25 of the information terminal 10 of the person B located far from the advertisement exhibit 39 has an advertisement exhibit corresponding to the distance from the advertisement exhibit 39 as indicated by reference numeral 44 in the figure. An information frame 46 is displayed together with a small captured image 45, and ““ attached information (discount coupon) is distributed as auxiliary information regarding the imaging of this advertising exhibit 39 inside the information frame 46. But it ’s too far away. Please approach within 30m from the advertisement. "" Is displayed.

  In this way, when the advertisement information of the captured image 45 projected on the display unit 25 can be read, the coupon is allowed to be acquired. When the advertisement information cannot be read, the distance is set when the coupon is acquired. It is possible to provide guidance for taking a picture at a reduced size. In other words, coupon information etc. can be distributed only to people (persons a) who can expect to read the details of the advertisement visually, while people (persons) who cannot read the details of the advertisement (B) can guide the user to shoot at a shorter distance. Here, the advertisement exhibit 39 is taken as an example, but the present invention is not limited to this. Any display medium may be used as long as it is used for advertisements, for example, a streetlight poster or a display. In addition, the display medium does not necessarily provide visible information, and may simply provide information by visible light communication.

  Here, the illumination light 40a from the lamp 40 is reflected by the advertisement exhibit 39, but the visible light communication style is not limited to this. For example, a backlight-type display panel, a backlight-type large display, a self-luminous display such as an LED, or the like may be used. In short, there is no particular limitation on the configuration as long as the modulation signal can be finally acquired using the information terminal 10.

  FIG. 12 is an explanatory diagram of information sent from the light source (lamp 40). In this figure, the information sent from the light source (lamp 40) includes a light source size information storage unit 47, a distribution maximum distance storage unit 48, and a distribution application storage unit 49. The light source size information is the projection size of the illumination light 40a emitted from the light source (lamp 40) to the advertisement exhibit 39, and corresponds to the size data 38b in FIG. This light source size information is given as an initial value when the lamp 40 is installed, but the maximum distribution distance is determined for each distribution information. That is, the maximum distribution distance is information for defining the maximum distance at which the character string of the advertisement exhibit 39 can be visually read. For example, the advertisement exhibit 39 can be replaced or the advertisement exhibit 39 Are updated by the maximum visually readable distance after the change or after the change.

  In the above description, the detection / capture preamble (see FIG. 6) included in the data format is a format required for the preamble * data body type. However, the signal detection / capture method is described above. As described above, there are various methods such as light source modulation by diffusing bit 0/1 to the pattern of the first pattern sequence SA / second pattern sequence SB. Since it has nothing to do with the essence, the illustration and description of the protocol part for signal detection / supplement such as preamble are omitted.

  FIG. 13 is a diagram illustrating a processing flow in an application example of a system including an advertisement exhibit and an information terminal having a camera function. In this figure, when the information (see FIG. 13) transmitted by the illumination light 40a of the lamp 40 is detected and received, first, distance range information (information stored in the maximum distribution distance storage unit 48) is extracted from the information. (Step S41), the distance to the information transmission point (the illumination light 40a on the advertisement exhibit 39) is calculated (Step S42). This distance calculation method is as described above.

  Next, it is determined whether or not the measurement distance is within the limited distance range, that is, whether or not the measurement distance is within the maximum delivery distance (step S43). As in 41, information that allows the coupon to be taken in is displayed (and stored or otherwise used) (step S44). On the other hand, if it is not within the maximum distribution distance, for example, as shown by reference numeral 44 in FIG. 11, information that does not allow the coupon to be taken in is displayed, and information indicating that the advertisement exhibit 39 is closer is displayed (step S45). .

  As described above, according to the application example to the special advertisement, the display or operation of the information terminal 10 is performed after referring to the distance condition in the information transmitted from the light source (lamp 40) as one of the processes in imaging. Can be determined. Therefore, there is an advertisement on the display medium, and information is transmitted through irradiation light (indirect light) that is further modulated in brightness, and information indicating the conditions of the information provision range is put on it, so the advertisement was actually confirmed visually It becomes possible to provide information linked to a very fine display medium in which coupon information and the like are distributed only to the user.

  In addition, it is possible to show "type of information that can be acquired" and "how close you can get", so that you can induce the user to read the advertisement in detail by guiding the user so that the information can be acquired It can be expected to enhance advertising effectiveness.

  In the above example, “if you are far”, the information is not acquired and guidance is given to reduce the distance. Conversely, “if you are close”, the information is not acquired and the distance is “Guidance to keep” may be used. This can be applied, for example, when it is desired to limit camera shooting at a short distance so that details can be understood in shooting a prototype product of an exhibition. Or it can be used to protect copyright and portrait rights.

  As a data format that can integrate such a case where it is desired to specify XXm or more and a case where it is desired to specify △ Δm or less, the distribution distance range may have an upper limit distance and a lower limit distance.

  FIG. 14 is an explanatory diagram of information having an upper limit distance and a lower limit distance. (A) is a structural diagram thereof, and (b) is an example of the stored information. The difference from the structure of FIG. 12 is that the delivery distance range storage unit 48 includes a lower limit distance storage unit 48a and an upper limit distance storage unit 48b. For example, when “0” is set in the lower limit distance storage unit 48a (unit is m; the same applies hereinafter) and “20” is set in the upper limit distance storage unit 48b, a delivery distance range within 20 m is designated. Alternatively, when “10” is set in the lower limit distance storage unit 48a and “0” is set in the upper limit distance storage unit 48b, a delivery distance range of 10 m or more is specified. Alternatively, when “5” is set in the lower limit distance storage unit 48a and “30” is set in the upper limit distance storage unit 48b, the delivery distance range from 5 m to 30 m is designated. Alternatively, when “0” is set for both the upper and lower limits, there is no distance limitation.

  Alternatively, various operations related to storage, editing, and transmission from shooting execution, such as permission / non-permission of shooting execution, may be controlled based on the distance information and the distance condition itself. This is exactly to protect copyrights and portrait rights, and “do not allow photography from nearby”, and conversely, to prevent taking photos that do not match the aesthetic sense of the subject, This is to enable photographing control from the subject side with a condition such as “permit photographing only within a range”.

  FIG. 15 is a diagram showing an example of a transmission format corresponding to permission / non-permission of photographing execution. As shown in this figure, the transmission format includes the same structure (light source size information storage unit 47, distribution distance range storage unit 48, and distribution application storage unit 49) as in the previous FIG. 12 or FIG. In addition, there is a difference in that the photographing operation restriction information storage unit 50 is included. In the photographing operation restriction information storage unit 50, both photographing and storage are possible (A), photographing is possible, but only the minimum resolution is permitted (B), and photographing is possible, but preservation is an image with watermark information. One of permission category classifications such as permission (C), photographing is allowed but saving is permitted with an image including a warning text character (D), and only monitor display is permitted (E) is set.

  In this way, it is possible to shoot and save, select the permitted category category from A to E, and set it in the shooting operation restriction information storage unit 50. It is possible to give fine restrictions such as allowing photography but saving with watermarked images, allowing photography but saving with warning text characters, and allowing only monitor display. In addition, here, restrictions are imposed on image display and storage, but in addition to these, restrictions may be imposed on secondary use (redistribution) of captured images by attaching e-mails or the like.

  In the case of shooting, the image itself also becomes a problem, so not only the distance condition but also the optical field angle information such as telephoto or wide angle (the angle of view information is necessarily obtained at the time of distance measurement). Further, it is more desirable to perform control with reference to image definition information.

<Specific application example 2>
FIG. 16 is a conceptual diagram when distance measurement data up to the light emitter 33 is used for focus control of the optical system 16. In this figure, as each process related to the imaging operation, the distance measurement data (distance D ′) to the light emitter 33 obtained as described in the above principle is used, and the imaging controller 19 and the central controller The autofocus control unit 18 is controlled via 14.

  Specifically, as long as the light emitter 33 is present at the target position of the subject 20 in FIG. 1 and the image 33a associated with the light emission of the light emitter 33 is picked up by the image pickup device 17, the inside of the viewfinder Regardless of the position of the light emitter 33, the distance to the light emitter 33 is measured according to the above explanation of the principle. Thus, if the image 33a of the light emitter 33 is captured somewhere in the viewfinder, the distance to the light emitter 33 can be measured and the optical system 16 can be focused. Thus, for example, there is no need to go through the troublesome procedure of returning to the original composition while the focus is locked, and pressing the shutter fully to perform shooting. By the way, in the application to autofocus, there is a possibility that the focus is not accurately adjusted during the distance measurement to the light emitting body 33. Therefore, the aperture of the photographing lens is set to the minimum (F value maximum). It is desirable to keep it. This is because, when the F value is maximized, the depth of field becomes deep and the pan focus is achieved (focusing from near to far).

<Specific application example 3>
FIG. 17 is a conceptual diagram in a case where required processing is performed on a captured image using distance measurement data up to the light emitter 33 as each process related to the imaging operation. In this figure, distance measurement data (distance D ′) to the light emitter 33 determined as described above is used to determine the distance to the light emitter 33, and sharpness processing is performed on the photographed image according to the determination result. Or apply bokeh processing. Here, sharpness processing is processing that clears the contour of an image, and blur processing is processing that reduces the sharpness of the contour. In general, it is better to sharpen the image by increasing sharpness for a subject located far away, while in the case of a subject located nearby, especially for portraits, such as rough skin It is said that it should be slightly blurred to prevent Ara from conspicuous. For example, in normal shooting techniques, when capturing a distant subject with a telephoto lens, the aperture is set as narrow as possible to achieve a clear image quality. A blur effect is actively produced by using a focus lens, a soft focus filter, a foggy filter, or the like.

  In the present embodiment, as shown in FIG. 17, sharpness processing and blur processing are performed on the captured image based on distance measurement data up to the light emitter 33, so that a special lens or filter is not used. You can get the image effect equivalent to such a normal shooting technique, so of course you can shoot an image with excellent aesthetics, even for beginners, easily using effective shooting techniques. be able to.

<Specific application example 4>
FIG. 18 is a conceptual diagram when distance measurement data up to the light-emitting body 33 is applied to control of photographing conditions, specifically to control of an optical system, as each process related to the imaging operation. In this figure, distance measurement data (distance D ′) to the light emitter 33 determined as described above is determined for each predetermined distance section. This distance division is, for example, a long distance, a medium distance, a short distance, a macro distance, or the like. When the photographing lens of the optical system 16 is a zoom lens with a macro function, the zoom lens is set to the telephoto (tele end of the minimum angle of view) if the determination result of the distance classification is a long distance, and the zoom lens if the distance is medium. Is set to the medium distance (intermediate angle of view), the zoom lens is set to the wide angle (wide end of the maximum angle of view) if it is a short distance, or the shortest shooting distance from the zoom lens if the determination result of the distance category is the macro distance Switch to a short macro lens. In this way, the angle of view of the photographic lens and the shortest photographic distance can be controlled according to the distance to the light emitter 33, so that the subject 20 can always be captured and photographed at the full angle of view of the lens. It is possible to obtain an excellent convenience in terms of convenience that it is not necessary to switch to a zoom lens magnification or macro.

It is a block diagram of the digital camera which shows an example of the application system of this embodiment. 3 is a configuration diagram of a light emitter 33. FIG. (A) is a figure which shows a part of storage space of RAM13 of the central control part 14 of the information terminal 10, (b) is a figure which shows some functional blocks implement | achieved by the central control part 14. FIG. 3 is a diagram illustrating an image formed on a light receiving surface 17a of the imaging device 17. FIG. It is a calculation conceptual diagram of the distance D. 3 shows an example of a data format 38. It is a figure which shows the flowchart in this Embodiment. It is a figure which shows the operation | movement flowchart of measurement process block S3. It is a specific explanatory view of a method for determining the measurement axis, weighting, and area of “image”. It is the conceptual diagram which shows an example of utilization of registration data. It is a conceptual diagram which shows the application example of the system which consists of an advertising exhibit and the information terminal provided with the camera function. It is explanatory drawing about the information sent out from a light source (lamp 40). It is a figure which shows the processing flow in the application example of the system which consists of an advertising exhibit and the information terminal provided with the camera function. It is explanatory drawing of the information which gave the upper limit distance and the lower limit distance. It is a figure which shows the example of the transmission format corresponding to permission / non-permission of imaging | photography execution. It is a conceptual diagram in the case of using distance measurement data to the light emitter 33 for focusing control of the optical system 16. It is a conceptual diagram in the case where required processing is performed on a photographed image using distance measurement data up to the light emitter 33. It is a conceptual diagram at the time of applying the ranging data to the light-emitting body 33 to control of an optical system.

Explanation of symbols

10 Digital camera (information terminal)
14 Central control unit 17 Imaging device 16 Optical system 33 Light emitter 40b Light emitter

Claims (8)

Imaging means;
An area detecting means for detecting an area having a predetermined luminance from an imaging surface imaged by continuously driving the imaging means;
Information acquisition means for acquiring information related to the size of the object corresponding to the area from the temporal luminance change in the area detected by the area detection means;
A distance calculating unit that calculates a distance between the device and the object based on the area of the region on the imaging surface, the imaging distance, and the information acquired by the information acquiring unit;
An imaging apparatus comprising: control means for controlling each process related to the imaging operation based on the distance calculated by the distance calculating means.   The control means reproduces the information acquired by the information acquisition means based on the distance between the device calculated by the distance calculation means and the object, or controls each process related to the imaging operation. The imaging apparatus according to claim 1, wherein storage is controlled. A display means,
The control means displays auxiliary information related to imaging on the display means based on the distance between the device calculated by the distance calculation means and the object as control of each process related to the imaging operation. The imaging apparatus according to claim 1, wherein the imaging apparatus is controlled as follows. Further comprising focusing means,
The control means controls the focusing means based on the distance between the device calculated by the distance calculation means and the object as control of each process related to the imaging operation. The imaging device according to claim 1.   The control means processes the image picked up by the image pickup means based on the distance between the device calculated by the distance calculation means and the object as control of each process related to the image pickup operation. The imaging apparatus according to claim 1, wherein the imaging apparatus is controlled.   The control means controls the optical system mechanism provided in the apparatus based on the distance between the apparatus and the object calculated by the distance calculation means as control of each process related to the imaging operation. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. An area detection step of detecting an area having a predetermined luminance from an imaging surface imaged by continuously driving the imaging unit;
An information acquisition step for acquiring information related to the size of the object corresponding to the region from the temporal luminance change of the luminance detected in the region detection step;
A distance calculating step of calculating a distance between the device and the object based on the area of the region on the imaging surface, the imaging distance, and the information acquired in the information acquiring step;
An imaging control method comprising: a control step for controlling each process related to the imaging operation based on the distance calculated in the distance calculating step. Computer
Area detecting means for detecting an area having a predetermined luminance from an imaging surface imaged by continuously driving the imaging unit;
Information acquisition means for acquiring information on the size of the object corresponding to the area from the temporal luminance change in the area detected by the area detection means;
A distance calculating means for calculating a distance between the device and the object based on the area of the region on the imaging surface, the imaging distance, and the information acquired by the information acquiring means;
An imaging control program that functions as a control unit that controls each process related to an imaging operation based on the distance calculated by the distance calculation unit.

Images