JP5413558B2 - Imaging apparatus, image processing method, and image processing program - Google Patents

Description

The present invention relates to an imaging device , an image processing method, and an image processing program that superimpose and combine information obtained from a two-dimensional code on a captured image.

  Conventionally, when taking a picture with a digital camera at a travel destination, it is convenient if the name of the shooting location, the hotel name of the accommodation, etc. can be combined with the captured image and stored. Also, when a product is photographed with a digital camera and posted on a web page for sale on the Internet, it is convenient if the name, nature, etc. of the product can be combined with the photographed product image.

  In order to solve such a problem, a digital camera capable of inputting character information has been proposed (see Patent Document 1). That is, a character input device such as a keyboard is provided in the digital camera, and characters such as a shooting record are input from the character input device such as a keyboard.

Further, when a character input device such as a keyboard is not provided in the digital camera itself, the captured image is taken into a personal computer, and image editing and character information input are performed using dedicated image editing software.
In addition, it has been proposed that a so-called cradle for supplying a charging voltage to a camera or transferring image information of the camera to another electronic device has a function of inputting characters or editing an image (Patent Document 2). reference).
JP 2003-009067 A JP 2003-169243 A

  However, in the digital camera disclosed in Patent Document 1, it is necessary to secure a space for arranging a plurality of keys, and if a character input device such as a keyboard for inputting character information is provided, the device is increased in size. There's a problem.

  When character information is synthesized using a personal computer and the cradle of Patent Document 2, it is difficult to immediately synthesize character information with a photographed image taken at a travel destination.

In view of the above-described problems, an object of the present invention is to provide an imaging device , an image processing method, and an image processing program that can easily synthesize information such as a character string on a captured image and can be downsized.

The imaging apparatus according to the first aspect of the invention includes an imaging unit, a first acquisition unit that acquires a first image including a two-dimensional code by the imaging unit, and the 2 acquired by the first acquisition unit. Creating means for creating a second image corresponding to a dimension code; second obtaining means for obtaining a third image including a subject different from the two-dimensional code by the imaging means; and the third image. Detecting means for detecting a feature point of the subject, and the second image is displayed superimposed on the position of the subject corresponding to the feature point in the third image, and the position or size of the subject. And a display control means for changing the position or size of the second image in accordance with the change of the second image .

An image processing method according to a second aspect of the present invention is an image processing method executed by an imaging apparatus having an imaging means and a display means, wherein the imaging means obtains a first image including a two-dimensional code. An acquisition step, a creation step of creating a second image corresponding to the two-dimensional code acquired by the first acquisition step, and a third object including a subject different from the two-dimensional code by the imaging means . A second acquisition step of acquiring an image; a detection step of detecting a feature point of the subject from the third image; and the second image as a subject corresponding to the feature point in the third image. And a display control step of changing the position or size of the second image in accordance with a change in the position or size of the subject. And butterflies.

According to a third aspect of the present invention, there is provided an image processing program comprising: a computer that controls an imaging apparatus having an imaging unit and a display unit; and a first acquisition unit that acquires a first image including a two-dimensional code by the imaging unit. A creation unit that creates a second image corresponding to the two-dimensional code acquired by the first acquisition unit and a third image including a subject different from the two-dimensional code are acquired by the imaging unit. A second acquisition unit, a detection unit for detecting a feature point of the subject from the third image, and the second image overlaid on the subject position corresponding to the feature point in the third image. And a display control unit that changes the position or size of the second image in accordance with a change in the position or size of the subject .

  According to the present invention, information such as a character string can be easily combined with a captured image. Further, since there is no need to provide a character input device such as a keyboard, the size can be reduced.

FIG. 1 is a perspective view showing an appearance of a digital camera 1 as an imaging apparatus according to an embodiment of the present invention.
As shown in FIG. 1A, a lens barrel 11 is disposed on the front panel of the digital camera 1.
As shown in FIG. 1B, a display 12 is disposed on the back panel of the digital camera 1. As the display 12, for example, an LCD (Liquid Crystal Display) is used. A shutter button 13 is provided on the upper surface of the digital camera 1. Various setting buttons 14 are arranged on the rear panel of the digital camera 1.

  FIG. 2 is a block diagram of the digital camera 1. As shown in FIG. 1, the digital camera 1 mainly includes an image data generation unit 100, a data processing unit 200, and a user interface unit 300.

  The image data generation unit 100 includes an optical lens device 101, an image sensor 102, and the like. The optical lens device 101 forms subject image light on the light receiving surface of the image sensor 102, and corresponds to the lens barrel 11 in FIG. The image data generation unit 100 is an imaging unit of this embodiment.

  The image sensor 102 photoelectrically converts subject image light formed on the light receiving surface. As the image sensor 102, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor is used. The output of the image sensor 102 is digitized and sent to the memory 201 of the data processing unit 200.

  The data processing unit 200 includes a memory 201, a video output device 202, an image processing device 203, a CPU (Central Processing Unit) 204, and a program code recording device 205.

  The memory 201 is used to temporarily store image data output from the image sensor 102, and to store data for performing various processes and image data to be recorded in an external storage device.

  The video output device 202 outputs the image data from the image sensor 102 to the outside via the user interface unit 300 and sends it to the display unit 301.

  The image processing device 203 performs various types of image processing. Image processing performed by the image processing apparatus 203 includes image data encoding / decoding processing, conversion processing from two-dimensional image data to display information, and processing of combining a captured image and character information. For example, JPEG (Joint Photographic Experts Group) is used as a still image encoding method. JPEG is a standard for image compression using discrete cosine transform. Of course, the still image encoding method is not limited to this. The image processing apparatus 203 is an information acquisition unit, a synthesis unit, a position acquisition unit, a position change unit, a size acquisition unit, a size change unit, and a display unit according to this embodiment.

  A CPU (Central Processing Unit) 204 controls the entire digital camera 1. The program code recording device 205 records a program for operating the CPU 204. In the present embodiment, the program of the program code recording device 205 includes a program that performs display information input processing, display information synthesis processing, and display position / size change processing. These processes will be described in detail later.

  The user interface unit 300 includes a display unit 301, an operation unit 302, a PC interface device 303, and an external storage device 304.

  The display unit 301 corresponds to the display 12 in FIG. 1 and is used as a viewfinder or the like at the time of shooting. The display unit 301 is used to monitor a captured image during reproduction.

  The operation unit 302 receives input from the user, and includes the shutter button 13 and various setting buttons 14 in FIG.

  The PC interface device 303 is connected to a personal computer, for example, and is used to transfer an image taken by a camera to the personal computer. For example, a USB (Universal Serial Bus) is used as the PC interface device 303.

  The external storage device 304 is used to store a captured image. As the external storage device 304, a card-type removable memory using a flash memory is used.

  In such a digital camera 1, during normal shooting, subject image light is imaged on the optical surface of the image sensor 102 by the optical lens device 101, and photoelectrically converted by the image sensor 102. The image signal is temporarily stored in the memory 201 after being digitized. In addition, the output of the image sensor 102 is sent from the video output device 202 to the display unit 301, and a screen during shooting is displayed on the display unit 301, and the display unit 301 functions as a viewfinder.

  When the shutter button 13 included in the operation unit 302 is pressed, the image data at that time is taken into the memory 201. The image data is encoded by, for example, the JPEG method by the image processing device 203 and stored in the external storage device 304.

FIG. 3 is a flowchart showing the operation of the digital camera 1 in the display information input mode.
When the display information input mode is set by operating the setting button 14, the CPU 204 reads out a program for executing the processing of the flowchart shown in FIG. 3 from the program code recording device 205. The CPU 204 executes display information input processing in the flowchart shown in FIG. 3 according to this program.

First, the CPU 204 detects whether or not the shutter button 13 has been half-pressed (step S101). If this determination is YES, the process moves to step S102, and if NO, the process returns to step S101.
In step S102, the image processing device 203 acquires a plurality of time-sequential images as the first image from the image data generation unit 100, and detects a two-dimensional code from the plurality of images. At this time, the position and size of the two-dimensional code in the acquired image are also acquired.
Next, the CPU 204 determines whether or not a two-dimensional code has been detected (step S103). If this determination is YES, the process proceeds to step S105, and if this determination is NO, the process returns to step S101.
In step S104, the information recorded in the two-dimensional code is read out by the image processing apparatus 203, and display information is acquired.

Next, based on the image acquired from the image data generation unit 100 by the image processing device 203, the position / size for changing the position or size of the display information following the change in the position or size of the two-dimensional code. The change process is executed. This position / size changing process will be described in detail later (step S105).
Next, in step S <b> 106, the display processing unit 203 superimposes the display information on the acquired image and displays it on the display unit 301. Here, the position and size of the display information are determined based on the position and size of the two-dimensional code in the acquired image. Specifically, the display information is displayed in the same position as the two-dimensional code.
In step S107, the CPU 204 detects whether the half-press of the shutter button 13 has been released. If this determination is YES, the process proceeds to step S109, and if NO, the process returns to step S105.
In step S <b> 108, display information is stored in the memory 201 by the image processing apparatus 203.

  For example, as shown in FIG. 4A, the user causes the lens barrel 11 of the digital camera 1 to face the two-dimensional code 21. In this two-dimensional code 21, character information is recorded.

In this state, when the user presses the shutter button 13 halfway, the two-dimensional code 21 is photographed by the digital camera 1, and the two-dimensional code 21 is displayed on the display 12 of the digital camera 1 as shown in FIG. An image G1 is displayed. Then, based on the image G1 of the two-dimensional code 21, character information G2 is acquired. As shown in FIG. 4C, the display 12 of the digital camera 1 is superimposed on the image G1 of the two-dimensional code 21. Character information G2 is displayed.
Here, the character information G2 is displayed over the image G1 of the two-dimensional code 21, but the present invention is not limited thereto, and the character information G2 may be displayed after the image G1 of the two-dimensional code 21 is deleted. .

  When the user releases the shutter button 13, the character information G2 is stored at the current position and size.

Hereinafter, the position / size changing process in step S105 will be described with reference to FIG.
First, it is detected whether or not the position of the two-dimensional code in the acquired image has changed (step S201). Specifically, the change in the position of the two-dimensional code is detected by detecting the position of the two-dimensional code for each of a plurality of continuous images in time series and comparing the position of the two-dimensional code in each image. To do.
If this determination is YES, the process proceeds to step S202, and if this determination is NO, the process proceeds to step S203.
In step S202, the image processing device 203 detects a change in the position of the two-dimensional code in the acquired image, and changes the position of the display information following the detected change amount.

Next, it is detected whether or not the size of the two-dimensional code in the acquired image has changed (step S203). Specifically, the size of the two-dimensional code is detected by detecting the size of the two-dimensional code for each of a plurality of continuous images in time series, and comparing the size of the two-dimensional code in each image. Detect changes.
If this determination is YES, the process proceeds to step S204, and if this determination is NO, the process ends.
In step S204, the image processing device 203 detects a change in the size of the two-dimensional code in the acquired image, and changes the size of the display information following the detected change amount.

  For example, when it is desired to change the display position of the character information G2, as shown in FIG. 6A, the user presses the shutter button 13 halfway while pressing the digital camera 1 with the two-dimensional code 21 indicated by arrows A1 and A2. Move in a direction along the surface. Then, since the position of the two-dimensional code 21 in the image captured by the digital camera 1 also changes, the image G1 of the two-dimensional code 21 on the display 12 follows the amount of change as shown in FIG. It moves from the position shown to the position shown in FIG.

  Next, when the user wants to change the size of the character information G2, as shown in FIG. 6B, the user presses the shutter button 13 halfway while holding the digital camera 1 with the two-dimensional code 21 indicated by the arrow A3. Move in a direction perpendicular to the surface. That is, the digital camera 1 is moved closer to or away from the surface of the two-dimensional code 21. Then, since the size of the two-dimensional code 21 in the image captured by the digital camera 1 also changes, the image G1 of the two-dimensional code 21 on the display 12 follows the amount of change, as shown in FIG. The size changes from the size shown to the size shown in FIG.

FIG. 8 is a flowchart showing the operation of the digital camera in the display information synthesis mode.
When the display information is already stored in the digital camera, the user sets the display information composition mode.
When the display information input mode is set by operating the setting button 14, the CPU 204 reads out a program for executing the processing of the flowchart shown in FIG. 8 from the program code recording device 205. The CPU 204 executes the display information synthesis process in the flowchart shown in FIG. 8 according to this program.

First, the stored display information is read out together with information about the position and size of the display information (step S301).
Next, the image processing apparatus 203 acquires a plurality of time-sequential images as the second image from the image data generation unit 100 (step S302).

Next, the read display information is superimposed on the acquired image and displayed on the display unit 301. Specifically, the display information is displayed at the position stored in the memory 201 in the size stored in the memory 201 (step S303).
Next, it is detected whether or not the shutter button 13 has been fully pressed (step S304). If this determination is YES, the process moves to step S305, and if NO, the process returns to step S302.
In step S <b> 305, the synthesized image is stored in the external storage device 304 by the image processing device 203.

  For example, as shown in FIG. 9, the user uses the digital camera 1 to take an image according to the subject 31. Then, as shown in FIG. 10, the character information G2 is read, and the image G3 including the subject 31 and the character information G2 are superimposed and displayed.

FIG. 11 is a flowchart showing the operation of the digital camera in the display position / size change mode.
When the display information input mode is set by operating the setting button 14, the CPU 204 reads out a program for executing the processing of the flowchart shown in FIG. 11 from the program code recording device 205. The CPU 204 executes the display position / size changing process in the flowchart shown in FIG. 11 according to this program.
In the display information composition mode described above, the position and size of the display information can be changed when the display information is stored. Even after the display information is stored, the display position / size change mode is set. The position and size of display information can be changed.

First, the CPU 204 selects and calls the stored display information (step S401).
Next, the CPU 204 detects whether or not the shutter button 13 is half-pressed (step S402). If this determination is YES, the process moves to step S403, and if NO, the process returns to step S402.
In step S403, the image processing device 203 acquires a plurality of time-sequential images as the third image from the image data generation unit 100, and detects feature points from the plurality of images.

Next, in step S404, it is determined whether or not a feature point has been detected. If this determination is YES, the process moves to step S405, and if NO, the process returns to step S402.
In step S405, the image processing device 203 detects the position and size of the feature point.

  Next, the image processing apparatus 203 executes position / size changing processing for changing the position and size of the display information following the change in the position or size of the feature point (step S406). In this position / size changing process, the same process as in step S107 described above is executed on the feature points instead of the two-dimensional code.

Next, the image processing apparatus 203 superimposes and combines this display information on the acquired image and displays it on the display unit 301 (step S407).
In step S408, the CPU 204 detects whether or not the half-press of the shutter button 13 has been released. If this determination is YES, the process proceeds to step S409, and if NO, the process proceeds to step S406.
In step S <b> 409, display information is stored in the memory 201 by the image processing apparatus 203.

  For example, as shown in FIG. 12, the user uses the digital camera 1 to take an image according to the subject 32. Then, the feature point of the subject 32 is extracted, and the character information G2 is displayed over the image G4 of the subject 32. Thereafter, as indicated by a broken line in FIG. 12, even when the user moves the digital camera 1, the relative positional relationship between the subject 32 and the character information G2 in the image G4 is maintained.

According to this embodiment, there are the following effects.
(1) The two-dimensional code 21 is imaged, and a plurality of time-series continuous images including the two-dimensional code 21 are acquired. Further, the subject 31 is imaged, and a plurality of time-sequential images G3 including the subject 31 are acquired. Then, the character information G2 is acquired based on the two-dimensional code included in the image, and the character information G2 is superimposed on the captured image G3 including the subject 31 and displayed. Therefore, information such as a character string can be easily combined with the captured image. Further, since there is no need to provide a character input device such as a keyboard, the size can be reduced.

(2) Changes in the position and size of the two-dimensional code 21 in the image obtained by capturing the two-dimensional code 21 are detected, and the position of the character information G2 in the image G3 is changed following the detected change amount. did.
Therefore, since the position and size of the display information can be changed only by changing the relative position of the digital camera with respect to the two-dimensional code 21, it is not necessary to use an external input device, and convenience can be improved.

(3) An arbitrary subject 32 including a feature point is imaged by the digital camera 1, a change in the feature point in the image G4 including the subject 32 is detected, and the amount of change detected is followed in the image G4. The position of the character information G2 is changed.
Therefore, since the position and size of the display information can be changed only by changing the relative position of the digital camera 1 with respect to the subject 32, it is not necessary to use an external input device, and convenience can be improved.

  It should be noted that the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within a scope that can achieve the object of the present invention are included in the present invention.

1 is a perspective view illustrating an appearance of a digital camera as an imaging apparatus according to an embodiment of the present invention. It is a block diagram of the digital camera of the embodiment. It is a flowchart which shows the operation | movement in the display information input mode of the digital camera of the said embodiment. It is a figure for demonstrating the procedure which inputs display information using the digital camera of the said embodiment. It is a flowchart which shows the position / size change process of the digital camera of the embodiment. FIG. 6 is a diagram (No. 1) for describing a procedure for changing the position and size of display information using the digital camera of the embodiment. FIG. 6 is a diagram (No. 2) for explaining the procedure for changing the position and size of display information using the digital camera of the embodiment. It is a flowchart which shows the operation | movement in the display information synthetic | combination mode of the digital camera of the said embodiment. FIG. 6 is a diagram (No. 1) for describing a procedure for synthesizing display information using the digital camera of the embodiment. It is FIG. (2) for demonstrating the procedure which synthesize | combines display information using the digital camera of the said embodiment. It is a flowchart which shows the operation | movement in the display position and magnitude | size change mode of the digital camera of the said embodiment. It is a figure for demonstrating the procedure which changes the position and magnitude | size of display information using the digital camera of the said embodiment.

Explanation of symbols

1 Digital camera (imaging device)
100 Image data generation unit (imaging means)
203 Image processing apparatus (information acquisition means, composition means, position acquisition means, position change means, size acquisition means, size change means, display means)

Claims (3)

Imaging means;
First acquisition means for acquiring a first image including a two-dimensional code by the imaging means;
Creating means for creating a second image corresponding to the two-dimensional code obtained by the first obtaining means;
Second acquisition means for acquiring a third image including a subject different from the two-dimensional code by the imaging means;
Detecting means for detecting feature points of the subject from the third image;
The second image is displayed so as to be superimposed on the position of the subject corresponding to the feature point in the third image, and the position of the second image is tracked following the change in the position or size of the subject. Or display control means for changing the size;
An imaging apparatus comprising: In an image processing method executed by an imaging apparatus having an imaging unit and a display unit,
A first acquisition step of acquiring a first image including a two-dimensional code by the imaging means;
A creation step of creating a second image corresponding to the two-dimensional code acquired by the first acquisition step;
A second acquisition step of acquiring a third image including a subject different from the two-dimensional code by the imaging means ;
A detection step of detecting a feature point of the subject from the third image;
The second image is displayed so as to be superimposed on the position of the subject corresponding to the feature point in the third image, and the position of the second image is tracked following the change in the position or size of the subject. Or a display control step of changing the size. A computer for controlling an imaging apparatus having an imaging means and a display means;
First acquisition means for acquiring a first image including a two-dimensional code by the imaging means;
Creating means for creating a second image corresponding to the two-dimensional code obtained by the first obtaining means;
Second acquisition means for acquiring a third image including a subject different from the two-dimensional code by the imaging means;
Detecting means for detecting feature points of the subject from the third image;
The second image is displayed so as to be superimposed on the position of the subject corresponding to the feature point in the third image, and the position of the second image is tracked following the change in the position or size of the subject. Alternatively, an image processing program which functions as a display control means for changing the size.

Images