JP4867515B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, and an image processing program, and more specifically, an image processing device, an image processing method, and an image that superimpose and display an image acquired by photographing or the like and information such as characters. It relates to a processing program.

  2. Description of the Related Art In recent years, as described in Patent Document 1, in a mobile phone having an imaging function, when data is registered in an address book, a person's face to be registered is photographed using the imaging function, and the photographing is performed. There are some images that can be registered in association with the address book data. Also, for address book data consisting of information such as characters, a person who registers in the address book immediately before or after shooting inputs characters by key operation or uses an infrared or other transmission means to register the address book data. A technique of receiving from a person's mobile phone is taken.

Regarding the transmission of information such as characters, in addition to the infrared rays, transmission by visible light communication technology is also conceivable. Visible light communication technology is LED (Light
Using a light-emitting element with excellent responsiveness (e.g., Emitting Diode), information is optically modulated and transmitted in a specific wavelength region of the visible light wavelength region, and on the receiving side, a light-receiving device or an imaging device is used. This is a general term for techniques for continuously photographing and demodulating the light-modulated area to restore information.

  For example, in the one described in Patent Document 2, a light emitting unit that transmits arbitrary data in a blinking pattern of light and an arbitrary subject are photographed using a light receiving unit using a camera-equipped mobile phone, The modulated light is emitted from the arrangement order of the light-on frame (the frame in which the light-emitting unit is turned on) and the light-off frame (frame in which the light-emitting unit is turned off) in each frame constituting the shot video. It describes that the data is demodulated and restored, and the reproduced data is superimposed on the through image on the monitor screen. That is, it is expected that the imaging process and the information input process to the address book data can be performed at once by using the visible light communication technique as described above.

JP 2003-244286 A (page 3 to page 4, FIG. 4) JP 2003-179556 A (pages 6 to 7, FIG. 6)

  However, when the visible light communication technique described in Patent Document 2 is used to create the address book data of Patent Document 1, the technique described in Patent Document 2 is, for example, substantially centered on the monitor screen. Since the position of the data display frame, such as the vicinity, is predetermined, when applied to registration of address book data, the image to be registered may be hidden behind the data display frame and become invisible. Conversely, if the imaging angle of view is adjusted to give priority to the display of the registration target image, the light emitting unit that transmits the light-modulated information may deviate from the imaging angle of view. And getting information is expected to be difficult.

  Therefore, the present invention provides an imaging apparatus, an image processing method, and an image processing method that can optimize the position of the display frame of the acquired information in accordance with the captured image when acquiring an image and information using the imaging function. An object of the present invention is to provide an image processing program.

In order to solve the above-mentioned problem, the invention according to claim 1 according to the first aspect is to set an imaging unit and a mode for giving priority to data transfer to the imaging unit, and start from an imaging region continuously captured. Detection means for detecting a region in which luminance changes in series, information acquisition means for acquiring information modulated from the luminance change region detected by the detection means, and demodulation for demodulating information acquired by the information acquisition means Means, control means for recognizing at least a partial area of the subject from the imaging area imaged by the imaging means when the mode giving priority to the data transfer is not set, and demodulated by the demodulation means A decision to select and determine an area in which the information to be superimposed is selected from an imaging area that avoids at least a partial area of the subject recognized by the control means And display means for displaying the stage, a state overlapped with the information the demodulated to the determined area by the determination unit to an image captured when the priority mode the data transfer is not set, the data Setting means for setting a plurality of areas in which the information demodulated by the demodulating means should be superimposed and displayed from the imaging area picked up by the imaging means when a mode giving priority to transfer is not set, and setting by the setting means Selecting means for selecting a specific area from the selected areas, wherein the determining means determines the area selected by the selecting means as an area on which the demodulated information is to be superimposed and displayed .
According to a second aspect of the present invention, in the first aspect of the invention, the recording instruction detecting means for detecting a recording instruction for an imaging region continuously imaged by the imaging means, and the recording instruction means When a recording instruction is detected, a recording control unit that controls to record the imaging area when the recording instruction is instructed, information demodulated by the demodulating unit, and a mode that prioritizes the data transfer are not set The image processing apparatus further includes storage means for storing the image recorded by the recording control means in association with each other.
According to a third aspect of the present invention, in the second aspect of the present invention, the recording control means further controls to set and record a mode that prioritizes image quality for the imaging region when the recording is instructed. It is characterized by doing.
According to a fourth aspect of the present invention, an imaging unit and a mode for giving priority to data transfer are set for the imaging unit, and detection is performed for detecting a region in which luminance changes in a time series from an imaging region continuously captured. Means for acquiring information modulated from the luminance change region detected by the detection means, a demodulation means for demodulating the information acquired by the information acquisition means, and a mode for giving priority to the data transfer. Control means for controlling to recognize at least a partial area of the subject from the imaging area imaged by the imaging means when not set, and an area on which the information demodulated by the demodulation means is to be superimposed and displayed A determination unit that selects and determines an imaging region that avoids at least a partial region of the subject recognized by the control unit; and a mode that prioritizes the data transfer. Display means for displaying a state in which the demodulated information is superimposed on an area determined by the determining means on an image picked up when no mode is set, and imaging continuously picked up by the imaging means When a recording instruction is detected by the recording instruction detecting means for detecting the recording instruction for the area and when the recording instruction is detected by the recording instruction detecting means, control is performed so as to set and record a mode that prioritizes image quality for the imaging area when the recording instruction is issued. Recording control means, and storage means for storing the information demodulated by the demodulation means and the image recorded by the recording control means in association with each other when the mode for giving priority to the data transfer is not set. It is characterized by having.
According to a fifth aspect of the present invention, in the invention according to any one of the second to fourth aspects, the recording control unit further includes at least a part of the subject in the imaging region when the recording is instructed. Further, the control is performed so that the predetermined color is corrected and recorded so as to be emphasized.
In order to solve the above-mentioned problem, the invention according to claim 6 according to the second aspect sets an imaging step and a mode for giving priority to data transfer to the imaging unit, and images are taken continuously in the imaging step. A detection step for detecting a region where luminance changes in time series from the imaging region, an information acquisition step for acquiring modulated information from the luminance change region detected in this detection step, and acquired in this information acquisition step A demodulating step for demodulating the information, and a control step for controlling to recognize at least a partial region of the subject from the imaging region imaged in the imaging step when a mode giving priority to the data transfer is not set. The area where the information demodulated in the demodulation step is to be superimposed and displayed is avoided, and at least a part of the subject recognized in the control step is avoided. A determination step of selecting and determining from the imaging region; and the demodulated information is superimposed on the region determined in the determination step on the image captured when the mode for prioritizing the data transfer is not set A display step for displaying and outputting the state, and an area in which the information demodulated in the demodulation step is to be superimposed and displayed from the imaging area captured in the imaging step when the mode giving priority to the data transfer is not set A plurality of setting steps, and a selection detecting step for detecting selection of a specific region from the regions set in the setting step, and the determining step includes a region in which selection is detected in the selection detecting step. The demodulated information is determined as a region to be superimposed and displayed .
According to a seventh aspect of the present invention, an imaging step and a mode in which data transfer is prioritized for the imaging unit are set, and an area in which luminance changes in time series from an imaging area continuously imaged in the imaging step. A detection step of detecting information, an information acquisition step of acquiring information modulated from the luminance change region detected in the detection step, a demodulation step of demodulating the information acquired in the information acquisition step, and the data A control step for controlling to recognize at least a partial region of the subject from an imaging region imaged in the imaging step when a mode giving priority to transfer is not set; and information demodulated in the demodulation step Determination to select and determine a region to be superimposed and displayed from an imaging region that avoids at least a partial region of the subject recognized in the control step A display step for displaying and outputting a state in which the demodulated information is superimposed on the area determined in the determination step on the image captured when the mode for giving priority to the data transfer is not set; A recording instruction detection step for detecting a recording instruction for an imaging area continuously imaged in the imaging step, and when a recording instruction is detected in the recording instruction detection step, the image quality for the imaging area when the recording instruction is issued A recording control step for controlling to record by setting a mode that prioritizes recording, information demodulated in the demodulation step, and recording in the recording control step when a mode for prioritizing the data transfer is not set And a storage step of storing the associated images in association with each other.
In order to solve the above-mentioned problem, an invention according to claim 8 according to a third aspect is that an imaging area in which a computer is set as an imaging means and a mode in which data transfer is prioritized for the imaging means is set, and images are continuously taken. Detecting means for detecting a region in which luminance changes in time series from, information acquiring means for acquiring information modulated from the luminance changing region detected by the detecting means, and demodulation for demodulating information acquired by the information acquiring means Means for controlling to recognize at least a partial area of the subject from the imaging area captured by the imaging means when the mode for giving priority to the data transfer is not set; information demodulated by the demodulation means Is selected and selected from an imaging area that avoids at least a part of the subject recognized by the control means. Determining means, display means for displaying a state overlapped with the information the demodulated to the determined area by the determination unit to an image captured when the priority mode the data transfer is not set, the data transfer Setting means for setting a plurality of areas in which the information demodulated by the demodulating means is to be superimposed and displayed from the imaging area picked up by the imaging means when the mode giving priority to the image is not set. The selecting means functions as a selecting means for selecting a specific area from the area, and the determining means determines the area selected by the selecting means as the area on which the demodulated information is to be superimposed and displayed .
According to the ninth aspect of the present invention, the computer is set with an image pickup means and a mode for giving priority to data transfer with respect to the image pickup means, and an area in which luminance changes in time series is detected from an image pickup area continuously picked up. Detecting means for detecting, information acquiring means for acquiring the modulated information from the brightness change area detected by the detecting means, demodulating means for demodulating the information acquired by the information acquiring means, and a mode for giving priority to the data transfer. Control means for recognizing at least a partial area of the subject from the imaging area imaged by the imaging means when not being performed, and an area for superimposing and displaying the information demodulated by the demodulation means Determining means for selecting and deciding from an imaging area that avoids at least a partial area of the subject recognized by the apparatus, giving priority to the data transfer Display means for displaying a state in which the demodulated information is superimposed on an area determined by the determining means on an image picked up when no mode is set, and an imaging area continuously picked up by the imaging means Recording instruction detecting means for detecting a recording instruction for the recording, and when the recording instruction is detected by the recording instruction detecting means, a recording control for controlling to set a mode giving priority to image quality for the imaging area when the recording instruction is given And a storage means for storing the information demodulated by the demodulation means and the image recorded by the recording control means in association with each other when the mode giving priority to the data transfer is not set. Features.

ADVANTAGE OF THE INVENTION According to this invention, when acquiring an image and information using an imaging function , the position of the display frame of the acquired information can be optimized according to a captured image.

  Embodiments of the present invention will be described below with reference to the drawings. However, the specification or examples of various details and the illustration of numerical values, character strings, and other symbols in the following description are for reference only to clarify the idea of the present invention, and all or part of them may Obviously, the idea of the invention is not limited. In addition, a well-known technique, a well-known procedure, a well-known architecture, a well-known circuit configuration, and the like (hereinafter, “well-known matter”) are not described in detail, but this is also to simplify the description. Not all or part of the matter is intentionally excluded. Such well-known matters are known to those skilled in the art at the time of filing of the present invention, and are naturally included in the following description.

  FIG. 1 is an external view of an example of a mobile phone 1 with an imaging function according to an embodiment of the present invention. In this figure, the mobile phone 1 has, for example, a foldable body 10, (a) shows a state where the body 10 is closed, and (b) shows a state where the body 10 is opened. The body 10 includes a lid portion 10a and a main body portion 10b. The lid portion 10a and the main body portion 10b are connected to each other by a hinge portion 10c so as to be foldable. A photographing lens 11, a light emitting unit 12, and a sub display unit 13 are provided on the surface of the lid 10 a (the surface exposed in the folded state). A main display unit 14 and a loudspeaker hole 15 are provided on the back surface (the surface hidden in the folded state) of the lid 10 a, and a speaker 16 is housed behind the loudspeaker hole 15. Various switches (power switch, on-hook key, off-hook key, cursor movement key, various function keys, numeric keys from 0 to 9, numeric keys for * and #, etc.) on the front surface (surface hidden in the folded state) of the main body 10b An operation element 17 and a transmission hole 18 are provided, and a microphone 19 is housed behind the transmission hole 18.

  FIG. 2 is an internal block diagram of the mobile phone 1 according to the embodiment. In this figure, a cellular phone 1 includes a CPU 20, a message memory 21, a modulation / light source driving unit 22, a light emitting unit 12, a display unit 23 (sub display unit 13 and main display unit 14), and a transmitter / receiver unit 24 (speaker 16 and microphone). 19), a user memory 25, a captured image memory 26, an imaging unit 27, an information light capturing and information decoding unit 28, a message / coordinate list memory 29, an operator 17 and the like.

  The CPU 20 controls the overall operation of the mobile phone 1. The modulation / light source drive unit 22 drives the light emitting unit 12 to emit light with a luminance change according to the control of the CPU 20, and transmits the address book data according to the luminance change pattern. For this data transmission by light, for example, when the data to be transmitted is binary data consisting of logic 0 and logic 1, one brightness change pattern series is assigned to logic 0 and two brightness change patterns are assigned to logic 1. A sequence may be assigned. These two luminance change pattern sequences can have different luminance change pattern sequences that have the same cycle and do not exist in nature (or have a low existence probability).

  When the mobile phone 1 is used as the information transmission side, the modulation / light source driving unit 22 sends its own address book data sent from the CPU 20 (character information such as name, organization name, address, telephone number, e-mail address; The light emitting unit 12 is driven in accordance with the character information registered in the owner information record 251a in FIG. 3, and the address book data is transmitted according to the luminance change pattern by the driving.

  The user memory 25 is a non-volatile storage device (for example, a semiconductor memory such as a flash memory) that includes an address book area 251, an image storage area 252, and other file storage areas 253. The address book area 251 is an area for holding owner information (owner's address book data) of the mobile phone 1 and a large number of address book data of others registered as necessary. The image storage area 252 is an area for storing an image file acquired by driving the imaging unit 27 with a storage destination address added thereto. The other file storage area 253 is an area for holding a file created by operating the mobile phone 1 itself.

  FIG. 3 is a conceptual structural diagram of the address book area 251. In this figure, the address book area 251 comprises one owner information record 251a and a large number of address book data records 251b, 251c, 251d, 251e, 251f,..., And the owner information record 251a includes The information of the owner of the mobile phone 1 is registered, and the address book data of the other person is registered in the address book data records 251b, 251c, 251d, 251e, 251f,. ing. Here, all of the owner information record 251a and the address book data records 251b, 251c, 251d, 251e, 251f,... Store character information such as name, organization name, address, telephone number, and e-mail address. Although the field 251g is secured, the address book data records 251b, 251c, 251d, 251e, 251f,..., In addition to the character information storage field 251g, further store the image file such as a face photo. A field 251i for storing an address is secured.

  The display unit 23 includes a sub display unit 13 and a main display unit 14. The sub display unit 13 displays a standby screen or the like of the mobile phone 1, and the main display unit 14 displays an incoming call screen or a call screen when the mobile phone 1 is used as a telephone, or displays the mobile phone 1 as an electronic device. When used as a camera, an electronic viewfinder screen for adjusting the shooting composition and a playback screen for shot images are displayed. Alternatively, a selection menu screen for using the mobile phone 1 for transmitting / receiving the address book data is displayed, a reproduction confirmation screen for the received address book data is displayed, and a setting screen for initial setting or the like is displayed if necessary. indicate.

  The operation element 17 includes various keys for turning on / off the power of the mobile phone 1 and dialing, and the mobile phone 1 is used as an electronic camera (imaging device) or as an information transmission device. In addition to a function selection key for performing an operation, a shutter key of an electronic camera is provided.

  The transmitter / receiver 24 includes a loudspeaker (speaker 16) and a sound collector (microphone 19) when the mobile phone 1 is used as a telephone. Note that the loudspeaker (speaker 16) is used for loudening shutter release sounds and various warning sounds in addition to this application.

  The imaging unit 27 is a two-dimensional image sensor composed of a CCD, a CMOS, or the like. In addition to being able to generate and output a high-resolution image (for example, an 1024 × 768-dot XGA standard image, although not particularly limited), the imaging unit 27 performs, for example, partial reading. The function can generate and output a low-resolution image of about 160 × 160 dots. The captured image memory 26 is a so-called image buffer that temporarily holds an image generated and output by the imaging unit 27, and at least one high-resolution image output from the imaging unit 27 is obstructed per buffer surface. It has enough capacity to store.

  The information light capturing and information decoding unit 28 sequentially captures periodic images of several tens of frames per second generated and output by the imaging unit 27 into the signal capturing frame buffers 28a, 28b, 28c, 28d,. Based on the image, the address book data transmitted in the light luminance change pattern is reproduced and restored. For example, when the light whose luminance is modulated on the transmission side is modulated with logic 0 as one luminance change pattern sequence and logic 1 as two luminance change pattern sequences, When detected, the pattern is replaced with logic 0, and when the second luminance change pattern series is detected, the pattern is replaced with logic 1, so that light modulated address book data, that is, name, organization name, address Play and restore text information such as phone numbers and e-mail addresses.

  The capacity of the signal capturing frame buffers 28a, 28b, 28c, 28d,... Per buffer surface may be smaller than the same capacity of the captured image memory 26. This is because the minimum necessary resolution for capturing the light source is, for example, about 160 × 120 dots. Incidentally, an image of 160 × 160 dots can be obtained by, for example, partial reading by the imaging unit 27 or interlaced reading with a hardware low-pass filter. Note that the imaging unit 27 according to the present embodiment can switch between a portrait shooting mode that prioritizes image quality and performs color correction that emphasizes skin color, and a data transfer priority mode that prioritizes frame rate over resolution. It is assumed that

  When the mobile phone 1 is used as a transmission side, the message memory 21 temporarily holds message data for transmission (stored in the owner information record 251a), and the message / coordinate list memory 29 The restored received data and coordinate information are retained.

Next, the operation will be described.
In the present embodiment, both the address book registration subject and the address book registrant own the mobile phone 1, and the address book registration subject operates the mobile phone 1 to operate the address book data transmission program. For example, the mobile phone is placed on the chest or the like so that it can be imaged together with the person to be registered. On the other hand, the registrant of the address book operates the mobile phone 1 owned by the address book to operate the “photograph with optical message” function, and the light emitting unit 12 of the mobile phone 1 possessed by the registration target person and the registration target person has the imaging field angle. Start shooting to enter.

  FIG. 4 is a diagram illustrating a flowchart of a transmission program executed by the mobile phone 1 owned by the person to be registered. In this figure, when the person to be registered operates the cellular phone 1 and the operator selects “output own address” in the menu (step S11), the CPU 20 stores the owner information record 251a in the address book area 251. Data (character information such as name, organization name, address, telephone number, and e-mail address) is read to create message data and output it to the message memory 21 (step S12). At this time, the person to be registered holds the mobile phone 1 at an appropriate position other than the face (preferably in front of the chest) with the surface on which the light emitting unit 12 of the mobile phone 1 is provided as the front.

  Next, the CPU 20 controls the modulation / light source driving unit 22 according to the information stored in the message memory 21 to drive the light emitting unit 12 with a luminance change. As a result, the information held in the message memory 21 is transmitted by the luminance change pattern of the light from the light emitting unit 12 (step S13). The registration target person continuously executes the above operation, and when the registration by the registrant is completed, an output end instruction is executed (step S14).

  FIG. 5 is a flowchart showing address book data reception / captured image display processing performed by the cellular phone 1 on the registrant side. In this figure, when "registration with optical message" in the menu is selected by the registrant (step S21), the CPU 20 sets the imaging unit 27 to the data transfer priority mode and sets the information light capture and information decoding unit 28 to the data transfer priority mode. Start (step S22).

  At this time, as described above, the registration target person holds the predetermined surface (the surface on which the photographing lens 11 is provided) of the mobile phone 1 as the front, and the registrant includes at least the face of the registration target person. The composition of the camera of the mobile phone 1 is adjusted so that the upper body and the mobile phone 1 held by the person to be registered are well within the shooting angle of view.

  During this time, the CPU 20 determines the end of data reception by causing the imaging unit 27 to continuously shoot light emission accompanied by a luminance change by the light emitting unit 12 at a predetermined period in the cellular phone 1 of the registration target person (step S23). If it is determined that reception has not been completed, a timeout is determined (step S24), and if the timeout occurs, a warning message (for example, "message not found") indicating that data could not be received for some reason is displayed (step After step S25), the process returns to step S21.

  On the other hand, if the end of data reception is determined, the CPU 20 next outputs a data reception end sound (step S26), and the previously received data is followed by a numeric string of a predetermined number of digits, or an @ mark. Whether or not the received data is address book data is determined based on whether or not a half-width alphanumeric string including dots and dots is included (step S27). If the address data is not included in the received data, an image file obtained by combining the received data is generated and stored in the image storage area 252 (step S28), and the flow ends. Is included, the image capturing unit 27 is switched from the data transfer priority mode to a portrait shooting mode in which image quality is prioritized and color correction for emphasizing the skin color is performed (step S29), and still image shooting and shooting sound ( A pseudo shutter sound is output (step S30).

  Next, the CPU 20 executes a captured image data superimposition process (step S31), and then adds a new record to the address book area 251, and stores the received address book data and image file storage address in the new record. Is registered (step S32), the data of the corresponding page is displayed on the main display unit 14 (step S33), and the flow is terminated.

  FIG. 6 is a diagram showing a flowchart of the data superimposing process in step S31 of FIG. In this data superimposing process, the address book data transmitted from the mobile phone 1 (light emitting unit 12) possessed by the person to be registered in step S22 of FIG. 5 is reproduced (step S311), and then the registrant's mobile phone. In the photographed image taken in step 1, the outline of the face is recognized as a part of the subject (step S312). As a technique for recognizing a contour portion, a region having a size corresponding to a shooting distance is cut out from a color input image, and a skin color region occupancy in the region is recognized as a face (contour) portion. Technology (see Japanese Patent Application Laid-Open No. 2005-78376) is known, and various other face recognition technologies (for example, Japanese Patent Application Laid-Open No. 5-297152) are also known. They can be selected or combined appropriately.

  In this way, when the contour portion of the face of the person to be registered is recognized as a part of the subject in the photographed image, next, an area avoiding the recognized contour portion is set as a data display frame candidate (step S313). . The shape of the data display frame is not particularly limited to this, but can be a shape imitating a so-called “balloon”.

  Next, it is determined whether or not the number of data display frame candidates set in step S313 is one (step S314). In one case, the received address book data (reproduced in step S311) is inserted into the data display frame and displayed superimposed on the photographed image (step S318), that is, in step S313. When there are a plurality of set data display frame candidates, the main display unit 14 displays all the plurality of candidates transparently (step S315).

  Thereafter, the registrant determines whether or not the data display frame has been selected by operating the operation element 17 (step S316). If not selected, for example, if the operation detection by the operation element 17 has not been detected for a predetermined time, the selection is made. Next, it is determined whether or not to cancel the display of the current data display frame from the presence or absence of a predetermined operation detected by the registrant operator 17 (step S317).

  If no cancel operation is detected, the display of the current data display frame is maintained and the process returns to step S315. If a cancel operation is detected, step S312 is performed again to perform face (contour) re-recognition processing. Return to the process. If a data display frame is selected in step S316, the received address book data is inserted into the data display frame and displayed superimposed on the captured image in step S318.

  FIG. 7 is a conceptual diagram of setting a data display frame in step S313. In this figure, a photographed image 30 includes an upper body image 32 including a face portion 31 of a registration target person, a telephone image 33 of a mobile phone 1 held by the registration target person, and a light emitting unit 12 of the mobile phone 1. A luminance change region 34 due to light emission is shown.

  Here, FIG. 7A shows two upper and lower data display frames 35 and 36 of the luminance change area 34, and FIG. Data display frames 37, 38, 39, 40 are shown. Among these data display frames, those that do not cover the face portion 31 are the lower data display frame 36 in FIG. 7A or the lower two data display frames 39 and 40 in FIG. 7B. .

  If the data display frame 36 as shown in FIG. 7A is set in step S313, the number of data display frames 36 set in this case is “one”, so that the user selection (step S315). ) Is passed, the address book data is inserted into the data display frame 36 and superimposed on the photographed image 30 (step S318). On the other hand, if the data display frames 39 and 40 as shown in FIG. 7B are set in step S313, the set data display frames 39 and 40 in this case are two candidate displays. After performing the determination processing of ˜317 to determine one of the data display frame candidates 39 and 40, the address book data is inserted into the determined data display frame (any one of the data display frames 39 and 40). The superimposed image is displayed on the captured image 30 (step S318).

  As described above, according to the present embodiment, the face portion of the person to be registered is recognized as a part of the subject in the photographed image, and the data display frame is set avoiding the face portion. It is possible to generate and display a superimposed face photographic image that is not hidden by the frame and that is not inconvenient in composition.

  In addition, when a plurality of data display frames are set as candidates, it is possible to arbitrarily select a data display frame at a preferred position in terms of composition, so that one of the data display frames can be determined by the registrant's operation. Can do.

  In this embodiment, the contour portion of the face of the registration target person is recognized as a part of the subject. However, the method for recognizing a part of the registration target person (subject) is not limited to this. For example, when photographing the whole body of a registration target person, by preparing a template that imitates a human figure, the whole body of the registration target person is imaged so as to match the template, and the area outside the template is displayed as data It can also be set as a frame.

  In the above embodiment, application to a mobile phone with a camera is taken as an example, but the present invention is not limited to this. Any portable electronic device with a camera may be used. For example, a portable information terminal or a digital camera may be used.

It is an external view of the mobile phone 1 in the embodiment, (a) is a state in which the body 10 is closed, (b) is a state in which the body 10 is opened. It is an internal block diagram of the mobile telephone 1 in embodiment. It is a conceptual structure figure of the address book area | region 251, (a) shows an owner information record, (b) shows the memory content of each address book data record. It is a figure which shows the flowchart of the transmission program run with the mobile telephone 1 by the side of a registration subject. It is a figure which shows the flowchart which shows the reception and captured image display process performed with the mobile telephone 1 of the registration side. It is a figure which shows the flowchart of the data superimposition process of FIG.5 S31. FIG. 4 is a conceptual diagram of setting a data display frame in step S313, where (a) shows a case where there is one data display frame candidate, and (b) shows a case where there are two data display frame candidates.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Cellular phone 12 Light emission part 17 Operator 20 CPU
23 Display unit 27 Imaging unit 35-40 Data display frame 251 Address book area

Claims (9)

Imaging means;
A detection unit that sets a mode for giving priority to data transfer with respect to the imaging unit, and detects a region in which luminance changes in time series from an imaging region that is continuously imaged;
Information acquisition means for acquiring modulated information from the luminance change region detected by the detection means;
Demodulation means for demodulating the information acquired by the information acquisition means;
Control means for controlling to recognize at least a partial area of the subject from the imaging area imaged by the imaging means when a mode giving priority to the data transfer is not set;
Determining means for selecting and determining an area in which the information demodulated by the demodulating means is to be superimposed and displayed from an imaging area that avoids at least a partial area of the subject recognized by the control means;
Display means for displaying a state in which the demodulated information is superimposed on an area determined by the determination means on an image captured when a mode for giving priority to the data transfer is not set ;
Setting means for setting a plurality of areas in which the information demodulated by the demodulating means is to be superimposed and displayed from the imaging area picked up by the imaging means when the mode giving priority to the data transfer is not set;
Selecting means for selecting a specific area from the area set by the setting means,
The determination unit determines an area selected by the selection unit as an area on which the demodulated information is to be superimposed and displayed . Recording instruction detection means for detecting a recording instruction for an imaging region continuously imaged by the imaging means;
When a recording instruction is detected by the recording instruction means, a recording control means for controlling to record the imaging area when the recording instruction is given;
And further comprising storage means for storing the information demodulated by the demodulation means and the image recorded by the recording control means in association with each other when the mode giving priority to the data transfer is not set. The imaging device according to claim 1. 3. The image pickup apparatus according to claim 2 , wherein the recording control unit further controls to set and record a mode that prioritizes image quality for the image pickup area when the recording is instructed . Imaging means;
A detection unit that sets a mode for giving priority to data transfer with respect to the imaging unit, and detects a region in which luminance changes in time series from an imaging region that is continuously imaged;
Information acquisition means for acquiring modulated information from the luminance change region detected by the detection means;
Demodulation means for demodulating the information acquired by the information acquisition means;
Control means for controlling to recognize at least a partial area of the subject from the imaging area imaged by the imaging means when a mode giving priority to the data transfer is not set;
Determining means for selecting and determining an area in which the information demodulated by the demodulating means is to be superimposed and displayed from an imaging area that avoids at least a partial area of the subject recognized by the control means;
Display means for displaying a state in which the demodulated information is superimposed on an area determined by the determination means on an image captured when a mode for giving priority to the data transfer is not set;
Recording instruction detection means for detecting a recording instruction for an imaging region continuously imaged by the imaging means;
When a recording instruction is detected by the recording instruction detection means, a recording control means for controlling to set and record a mode that prioritizes image quality for the imaging area when the recording instruction is given;
Storage means for associating and storing the information demodulated by the demodulation means and the image recorded by the recording control means when the mode giving priority to the data transfer is not set;
An imaging apparatus comprising: Said recording control means further claims 2 to 4 for at least a partial region of the object, and controls to record to correct to emphasize the predetermined color in the imaging area when the recorded instructions The imaging device according to any one of the above. Imaging step;
A detection step of setting a mode for giving priority to data transfer to the imaging unit, and detecting an area in which luminance changes in time series from an imaging area continuously imaged in the imaging step;
An information acquisition step of acquiring information modulated from the luminance change region detected in this detection step;
A demodulation step for demodulating the information acquired in this information acquisition step;
A control step for controlling to recognize at least a partial region of the subject from the imaging region imaged in the imaging step when a mode giving priority to the data transfer is not set;
A determination step of selecting and determining an area in which the information demodulated in the demodulation step is to be superimposed and displayed from an imaging area that avoids at least a partial area of the subject recognized in the control step;
A display step for displaying and outputting a state in which the demodulated information is superimposed on the area determined in the determination step on an image captured when a mode for giving priority to the data transfer is not set ;
A setting step for setting a plurality of areas in which the information demodulated in the demodulation step is to be superimposed and displayed from the imaging area captured in the imaging step when the mode for giving priority to the data transfer is not set;
A selection detecting step for detecting selection of a specific region from the region set in this setting step,
The determining step determines an area where selection is detected in the selection detecting step as an area where the demodulated information is to be superimposed and displayed . Imaging step;
A detection step of setting a mode for giving priority to data transfer to the imaging unit, and detecting an area in which luminance changes in time series from an imaging area continuously imaged in the imaging step;
An information acquisition step of acquiring information modulated from the luminance change region detected in this detection step;
A demodulation step for demodulating the information acquired in this information acquisition step;
A control step for controlling to recognize at least a partial region of the subject from the imaging region imaged in the imaging step when a mode giving priority to the data transfer is not set;
A determination step of selecting and determining an area in which the information demodulated in the demodulation step is to be superimposed and displayed from an imaging area that avoids at least a partial area of the subject recognized in the control step;
A display step for displaying and outputting a state in which the demodulated information is superimposed on the area determined in the determination step on an image captured when a mode for giving priority to the data transfer is not set;
A recording instruction detection step for detecting a recording instruction for an imaging region continuously imaged in the imaging step;
When a recording instruction is detected in this recording instruction detection step, a recording control step for controlling to set and record a mode that prioritizes image quality for the imaging area when the recording instruction is given;
A storage step of associating and storing the information demodulated in the demodulation step and the image recorded in the recording control step when a mode giving priority to the data transfer is not set;
An image processing method comprising: Computer
Imaging means,
A detection unit that sets a mode for giving priority to data transfer to the imaging unit, and detects a region in which luminance changes in time series from an imaging region that is continuously captured,
Information acquisition means for acquiring modulated information from the luminance change region detected by the detection means;
Demodulation means for demodulating the information acquired by the information acquisition means;
Control means for controlling to recognize at least a partial area of the subject from an imaging area imaged by the imaging means when a mode giving priority to the data transfer is not set;
Determining means for selecting and determining an area in which the information demodulated by the demodulating means is to be superimposed and displayed from an imaging area that avoids at least a partial area of the subject recognized by the control means;
Display means for displaying a state in which the demodulated information is superimposed on a region determined by the determination means on an image captured when a mode giving priority to data transfer is not set;
Setting means for setting a plurality of areas in which the information demodulated by the demodulating means is to be superimposed and displayed from the imaging area picked up by the imaging means when the mode giving priority to the data transfer is not set;
Function as a selection means for selecting a specific area from the area set by the setting means,
The determining means determines an area selected by the selecting means as an area on which the demodulated information is to be superimposed and displayed. Computer
Imaging means,
A detection unit that sets a mode for giving priority to data transfer to the imaging unit, and detects a region in which luminance changes in time series from an imaging region that is continuously captured,
Information acquisition means for acquiring modulated information from the luminance change region detected by the detection means;
Demodulation means for demodulating the information acquired by the information acquisition means;
Control means for controlling to recognize at least a partial area of the subject from an imaging area imaged by the imaging means when a mode giving priority to the data transfer is not set;
Determining means for selecting and determining an area in which the information demodulated by the demodulating means is to be superimposed and displayed from an imaging area that avoids at least a partial area of the subject recognized by the control means;
Display means for displaying a state in which the demodulated information is superimposed on a region determined by the determination means on an image captured when a mode giving priority to data transfer is not set;
Recording instruction detection means for detecting a recording instruction for an imaging region continuously imaged by the imaging means;
When the recording instruction is detected by the recording instruction detecting means, a recording control means for controlling to set and record a mode that prioritizes image quality for the imaging area when the recording instruction is given,
Storage means for storing the information demodulated by the demodulation means and the image recorded by the recording control means in association with each other when the mode giving priority to the data transfer is not set;
An image processing program that functions as an image processing program.

Images