JP5007631B2 - Electronic camera - Google Patents

Description

The present invention relates to an electronic camera.

  Conventionally, various methods for searching for desired information from various information such as a captured image recorded in a camera have been proposed. Patent Document 1 discloses an electronic camera that, when a user inputs a keyword such as a file name, a shooting date, and a name, searches for a corresponding image from recorded images based on the keyword. .

JP 2006-166193 A

  The electronic camera disclosed in Patent Document 1 is troublesome for the user because a search keyword must be input. Therefore, there is a need for an electronic camera that can easily search for and display desired information without inputting a keyword.

The electronic camera according to claim 1 is based on an input unit that inputs information including data relating to a position, an imaging unit that captures an image of a subject, a position detection unit that detects a position, and a position detected by the position detection unit. A search unit that searches for the information, a display unit that displays the information searched by the search unit together with an image captured by the imaging unit, and a tilt of the electronic camera in accordance with data relating to the position of the information. An inclination detection unit for detecting, the information includes time data at which the information is generated, and the search unit generates the information to be searched according to the inclination detected by the inclination detection unit. It is characterized by changing the time range.
The electronic camera according to claim 5 is based on an input unit that inputs information having data regarding a position, an imaging unit that captures an image of a subject, a position detection unit that detects a position, and a position detected by the position detection unit. A search unit that searches for the information, and a display unit that displays the information searched by the search unit together with an image captured by the imaging unit according to data relating to a position of the information, and the image Is a through image, and the display unit displays a background image stored in advance by switching from the through image when a predetermined switching condition is satisfied.
The electronic camera according to claim 8 is based on an input unit that inputs information including data relating to a position, an imaging unit that captures an image of a subject, a position detection unit that detects a position, and a position detected by the position detection unit. A search unit that searches for the information, and a display unit that displays the information searched by the search unit together with an image captured by the imaging unit according to data relating to a position of the information, and the display The unit displays a line indicating a pseudo three-dimensional space together with the image.

  According to the present invention, desired information can be easily retrieved and displayed.

  The electronic camera of the present invention will be described below with reference to the drawings. FIG. 1 is a configuration diagram of a camera system including a digital camera according to an embodiment of the present invention. The camera system shown in FIG. 1 includes a digital camera 1, an access point 2, and a server device 3. The access point 2 and the server device 3 are connected via the Internet.

  The digital camera 1 is an electronic camera that takes a subject in accordance with a user's shooting instruction and acquires captured image information. The digital camera 1 also has a wireless communication function for performing wireless communication with the access point 2. By this wireless communication, the digital camera 1 and the server device 3 are connected via the access point 2 and the Internet. The configuration of the digital camera 1 will be described in detail later with reference to FIGS.

  The access point 2 is a device for enabling various devices including the digital camera 1 having a wireless communication function to communicate using a network such as the Internet. When the digital camera 1 and the access point 2 perform wireless communication, the digital camera 1 and the server device 3 are connected as described above.

  The server device 3 stores a database of information called a space-time database. In this space-time database, various information is recorded in association with the acquisition position and acquisition time of the information. For example, photographic image information acquired by the digital camera 1 or another camera, character information, audio information, and the like are recorded in the space-time database in association with the acquisition position and acquisition time.

  Next, the configuration of the digital camera 1 will be described. FIG. 2 is a diagram illustrating an appearance of the digital camera 1. 2, the digital camera 1 includes a shutter button 11, a power button 12, a zoom button 13, a playback button 14, a menu button 15, a delete button 16, a jog dial 17, a determination button 18, and a LAN (Local Area Network) access. LED (Light Emitting Diode) 19, strobe charging LED 20, AF (Auto Focus) LED 21, memory access LED 22, power LED 23, and monitor 104.

  The shutter button 11 is an operation button for the user to give a shooting instruction to the digital camera 1 and has a two-stage switch of half-press and full-press. The power button 12 is an operation button for the user to switch the main power source of the digital camera 1 from on to off or from off to on. The zoom button 13 is an operation button used when the user changes the focal length of the photographing lens of the digital camera 1.

  The playback button 14 is an operation button for the user to instruct the digital camera 1 to play back image data. The menu button 15 is an operation button for the user to instruct the digital camera 1 to call a menu screen. The delete button 16 is an operation button used when the user deletes image data from the memory card loaded in the digital camera 1.

  The jog dial 17 is a rotatable operation member for a user to give an instruction to move the cursor on the screen. The determination button 18 is an operation button for the user to determine setting contents on the menu screen.

  The LAN access LED 19 is a display device that lights up when the digital camera 1 is performing wireless communication with the access point 2. When the LED 19 for LAN access is turned on, the user can be notified that the digital camera 1 is communicating. The strobe charging LED 20 is a display device that is lit while the strobe is being charged. When the strobe charging LED 20 is turned on before photographing, the user can be informed that the strobe is being charged and preparation for photographing is not completed.

  The AF LED 21 is a display device for informing the user of the AF in-focus state. The memory access LED 22 is a display device that lights up in a memory access state, that is, when data is being written to a memory card loaded in the digital camera 1 and when data is being read from the memory card. The power LED 23 is a display device that lights up when the digital camera 1 is in a power-on state.

  The monitor 104 is a color display device using an LCD (Liquid Crystal Display) or the like, and displays various images. For example, when the monitor 104 displays a through image, the monitor 104 is used as a viewfinder at the time of shooting. The through image (through image) is a photographed image of a subject displayed on the monitor 104 in real time when an operation mode such as a photographing mode or an information search mode described later is set in the digital camera 1. is there. Further, the monitor 104 can also display a captured image, display a menu screen, and the like.

  FIG. 3 is a block diagram of the digital camera 1. As shown in this block diagram, the digital camera 1 includes a CPU 101, a compression / expansion circuit 102, a display driver 103, a monitor 104, a key input unit 105, a wireless LAN circuit 106, a GPS (Global Positioning System) sensor 107, and a motion sensor 108. , Capacitor 109, battery 110, strobe 111, CCD (Charge Coupled Device) driver 112, CCD 113, and memory 114. A memory card 115 as a recording medium is loaded in a card slot (not shown).

  The CPU 101 is a circuit for executing various processes and controls, and is configured using a microcomputer or the like. The compression / decompression circuit 102 is a circuit that compresses / decompresses captured image data. The display driver 103 is a drive circuit that controls an image displayed on the monitor 104. The key input unit 105 is a circuit that detects operations of buttons, switches, dials, and the like provided in the digital camera 1.

  The wireless LAN circuit 106 is a communication circuit for communicating with an external device wirelessly. By performing wireless communication with the access point 2 by the wireless LAN circuit 106, the digital camera 1 is connected to the server apparatus 3 via the Internet.

  The GPS sensor 107 receives a GPS signal transmitted from a GPS satellite and outputs it to the CPU 101. Based on this GPS signal, the CPU 101 detects the current position of the digital camera 1. The motion sensor 108 detects the posture change of the digital camera 1 in each direction of pitch, yaw, and roll. Note that when the optical axis of the digital camera 1 is oriented in the horizontal direction, the change in posture in the pitch direction represents a change in posture around an axis that is orthogonal to the optical axis, that is, a change in elevation angle. Represents a change in posture around an axis perpendicular to the optical axis, that is, a change in azimuth. Further, the posture change in the roll direction represents a posture change around the optical axis, that is, a change in the tilt of the digital camera 1.

  The capacitor 109 is charged by receiving electric power from the battery 110 and is used for light emission of the strobe 111. It is also used to drive the camera as a backup power source in case of emergency. The battery 110 is a power source such as a lithium ion rechargeable battery that supplies power necessary for operating the digital camera 1.

  The strobe 111 enables photographing even in a dark environment by irradiating the subject with auxiliary light. The CCD driver 112 is a circuit for driving the CCD 113. The CCD 113 is an image sensor for capturing a subject image via a photographing lens. When a subject image is picked up by the CCD 113, an image pickup signal is output from the CCD 113 to the CCD driver 112. The imaging signal is converted into captured image data by the CCD driver 112 and then output to the CPU 101.

  The memory 114 is a nonvolatile semiconductor memory, and stores programs, data, and the like for use in control executed by the CPU 101. The memory card 115 stores various information such as photographed image information acquired by the digital camera 1 under the control of the CPU 101. Note that information is recorded in the memory card 115 in association with the acquisition position and the acquisition time, as in the server device 3 of FIG. 1 described above. That is, the memory card 115 stores a space-time database based on information acquired by the digital camera 1.

  Next, the operation of the digital camera 1 will be described. The digital camera 1 can set various operation modes such as a shooting mode for acquiring a shot image and an information search mode for searching for information from a space-time database according to a user operation.

  When the shooting mode is set, the digital camera 1 acquires a through image as described above based on the image of the subject imaged by the CCD 113 and displays it on the monitor 104 before imaging the subject. . When a shooting instruction is performed by the user operating the shutter button 11, the digital camera 1 captures a subject and acquires a captured image in accordance with the shooting instruction, and stores information on the acquired captured image in the memory card 115. Record.

  When recording photographed image information in the memory card 115, the digital camera 1 adds various information at the time of photographing as meta information to the photographed image information. For example, various information such as the shooting time, the angle of view of the shot image, the focal length, the shutter speed, the position of the main subject, and the distance to the main subject are added as meta information to the shot image information and recorded in the memory card 115. . The meta information includes position information of the digital camera 1 detected at the time of shooting using the GPS sensor 107. That is, the image picked up by the CCD 113 has information (data) related to the position detected by the CPU 101 based on the GPS signal output from the GPS sensor 107 as meta information. The captured image information to which such meta information is added is recorded in the memory card 115, so that the information of the captured image acquired by the digital camera 1 is associated with the acquisition position and acquisition time of the memory card 115. Recorded in space-time database.

  Further, at this time, the digital camera 1 detects the orientation of the digital camera 1 at the time of shooting based on the posture change of the digital camera 1 detected by the motion sensor 108. At this time, the orientation of the digital camera 1 at the time of shooting is detected by obtaining an azimuth angle representing the horizontal direction of the digital camera 1 and an elevation angle representing the vertical direction of the digital camera 1. The captured image information may be further associated with the acquisition direction and recorded in the space-time database of the memory card 115 by adding the detected shooting direction information to the meta information.

  A part of the captured image information recorded in the memory card 115 may be transmitted from the digital camera 1 to the server device 3 in accordance with a user operation and recorded in the space-time database in the server device 3. At this time, information such as the position, orientation, and time of the digital camera 1 detected when the captured image is acquired is transmitted as meta information together with the captured image information. By recording the meta information together with the captured image information, the captured image information is recorded in the space-time database of the server device 3 in association with the acquisition position, the acquisition time, the acquisition direction, and the like. That is, in the space-time database of the memory card 115 or the server device 3, information having meta information such as position data, direction data, time data when the information is created, and the like is recorded.

  On the other hand, when the information search mode is set, the digital camera 1 searches and extracts information satisfying a predetermined search condition from the space-time database stored in the server device 3 or the memory card 115. Then, the extracted information is displayed on the monitor 104 superimposed on the through image and presented to the user. Information extracted from the space-time database of the server device 3 is transmitted to the digital camera 1 via the Internet and the access point 2.

  The digital camera 1 can set a search target area for an information acquisition position as one of search conditions when searching for information from a space-time database. This search target area includes the current position (camera position) of the digital camera 1 based on the GPS signal received by the GPS sensor 107 and the optical axis direction (camera direction) based on the attitude of the digital camera 1 detected by the motion sensor 108. ). Specifically, the current camera position is set as the viewpoint, and a vector having a predetermined length extending from the viewpoint in the direction of the camera is set as a line-of-sight vector. A predetermined spatial range centered on this line-of-sight vector is set as the search target area. Note that the search target area is set to become wider as the distance from the viewpoint increases. Or the opposite may be sufficient.

  In setting the search target area, only the horizontal camera direction, that is, the azimuth angle may be used. That is, a predetermined spatial range can be set as the search target area for the horizontal direction, and the entire spatial range can be included in the search target area for the vertical direction. Alternatively, in addition to the horizontal camera direction, the vertical camera direction, that is, the elevation angle may also be used to set a predetermined spatial range in the search target area in each of the horizontal direction and the vertical direction. Furthermore, the search target area may be set based only on the camera position without using the camera orientation. That is, a predetermined range centered on the current camera position may be set as the search target area.

  In setting search conditions, in addition to the search target area as described above, a search target time for information acquisition time (creation time) may be set. That is, a predetermined time range is set from the past to the present, and the search condition is set by setting the time range as the search target time. The range of the search target time is set as follows based on the tilt around the optical axis (camera tilt) of the digital camera 1 detected by the motion sensor 108.

  When the current camera tilt is almost zero, that is, when the user is holding the digital camera 1 almost straight, a predetermined time range from the present time, for example, a time range up to one year ago is set as the search target time. To do. When the user tilts the digital camera 1 clockwise from that state, the range of the search target time is gradually changed back to the past. At this time, as the camera tilt angle increases, the range of the search target time may be changed greatly accordingly. For example, three stages of threshold values of 10 °, 20 ° and 30 ° are set for the camera tilt angle. When the camera tilt angle is 10 ° or more and less than 20 °, the search target time is set every month, every month when it is 20 ° or more and less than 30 °, and every year when it is 30 ° or more. Change the range retroactively. Conversely, when the user tilts the digital camera 1 counterclockwise, the range of the search target time is gradually changed from the past to the present. Similarly, the search target time can be changed greatly as the camera tilt angle increases.

  When the search conditions as described above are set in the digital camera 1, information satisfying the search conditions is extracted from the space-time database stored in the server device 3 or the memory card 115. That is, information related to the acquisition position in the search target area set as the search condition and information related to the acquisition time (creation time) within the search target time set as the search condition are searched from the space-time database. To be extracted. At this time, the space-time database of the memory card 115 is searched first, and if the corresponding information is not extracted, the space-time database of the server device 3 may be searched. Alternatively, both the memory card 115 and the space-time database of the server device 3 may be searched. Note that when searching the space-time database of the server device 3, the search conditions set in the digital camera 1 are transmitted to the server device 3 via the access point 2 and the Internet. Information extracted from the space-time database of the server device 3 is transmitted to the digital camera 1 via the Internet and the access point 2.

  The digital camera 1 displays the information extracted as described above from the space-time database of the server device 3 or the memory card 115 on the monitor 104 so as to be superimposed on the through image. Thereby, the information retrieved from the space-time database and extracted is presented to the user.

  Alternatively, information recorded in the space-time database may be input to the CPU 101 of the digital camera 1 and the information may be searched for in the digital camera 1. For example, information corresponding to a predetermined range from the current position of the digital camera 1 is transmitted from the space-time database to the digital camera 1 and stored in the digital camera 1. From this information, information satisfying the search condition set as described above may be searched and displayed together with the through image.

  Note that the information retrieved from the space-time database is not limited to the information of the captured image acquired by the digital camera 1. For example, information on a photographed image by another camera, character information other than the photographed image, voice information, and the like may be recorded in a space-time database and retrieved. Any information that is recorded in the space-time database in association with the acquisition position and the acquisition time can be searched. That is, the CPU 101 can search by inputting various information having data regarding the position from the space-time database.

  FIG. 4 is an example of a through image displayed on the monitor 104 when the shooting mode is set. When the shutter button 11 is not pressed in the shooting mode, such a through image is displayed. On the other hand, FIG. 5 is an example of a display screen on the monitor 104 when the information search mode is set. Hereinafter, such a display screen is referred to as a browser view screen.

  In the browser view screen of FIG. 5, grid lines for providing a three-dimensional visual effect are displayed so as to overlap the through image of FIG. Thereby, a pseudo three-dimensional space (hereinafter referred to as a pseudo three-dimensional space) is expressed on the screen. Further, a space cursor 30 is displayed in a predetermined area of the pseudo three-dimensional space.

  The range of the pseudo three-dimensional space on the browser view screen is determined according to the above-described camera position and camera orientation. That is, the digital camera 1 uses the current camera position detected based on the GPS signal received by the GPS sensor 107 as a viewpoint, and looks at the current camera direction based on the posture of the digital camera 1 detected by the motion sensor 108. A display target range in real space is set as a vector. The display target range at this time is preferably the same size as or smaller than the above search target area so that all the information extracted from the space-time database can be displayed. The pseudo three-dimensional space corresponding to the display target range set in this way is superimposed on the through image and expressed on the screen of the monitor 104. Thus, the range of the pseudo three-dimensional space on the browser view screen is set corresponding to the display target range in the real space according to the current camera position and camera orientation. Therefore, the range of the pseudo three-dimensional space is coincident with the display range of the through image.

  On the browser view screen, the space cursor 30 freely moves in the pseudo-stereoscopic space in accordance with the change in posture of the digital camera 1 detected by the motion sensor 108. For example, the digital camera 1 moves in the left-right direction on the plane on which the grid lines are drawn in accordance with a change in posture in the yaw direction, that is, a change in azimuth angle. In addition, the digital camera 1 moves in the depth direction on the plane on which the grid lines are drawn in accordance with a change in posture in the pitch direction, that is, a change in elevation angle. Accordingly, the user can intuitively operate the digital camera 1 to move the space cursor 30 as desired in the pseudo-stereoscopic space.

  Note that the space cursor 30 may be further moved in the vertical direction in the pseudo three-dimensional space. For example, the spatial cursor 30 is moved in the vertical direction in accordance with the change in posture of the digital camera 1 in the pitch direction. At this time, the movement in the depth direction is performed according to the operation of the zoom button 13, for example. Alternatively, the space cursor 30 may be moved in accordance with the operation of the jog dial 17 or the operation of a cross key (not shown) regardless of the posture change of the digital camera 1.

  In the browser view screen of FIG. 5 described above, icons 31 to 36 representing information extracted from the space-time database are respectively displayed at predetermined positions in the pseudo three-dimensional space. The display position of each icon is determined corresponding to the acquisition position of information represented by each icon, that is, the acquisition position recorded in the space-time database in association with the information. In other words, the icons 31 to 36 are respectively displayed at positions in the pseudo three-dimensional space corresponding to the acquisition positions of the information in the real space. That is, each piece of information represented by the icons 31 to 36 is arranged at a position in the through image corresponding to the position data of the meta information included in each piece of information.

  The icons 31 to 34 all represent captured image information, that is, images captured by the CCD 113 of the digital camera 1 or an image sensor of another camera. In the captured images of the icons 31 and 34, a mountain is photographed as the main subject. In the captured image of the icon 32, a person's face is photographed as the main subject. A tree is photographed as a subject. Each of these icons representing captured image information is displayed as a three-dimensional icon (three-dimensional icon). On the other hand, an icon 35 and an icon 36 displayed in a planar shape represent information other than the captured image information. The icon 35 represents character information, and the icon 36 represents voice information.

  The icon 31 is a rectangular parallelepiped solid icon, and a thumbnail image obtained by reducing a captured image is pasted on one surface. The image pasted on the three-dimensional surface in this way is called a texture. The icon 31 represents the shooting condition in the shot image by this rectangular parallelepiped. For example, the direction of the cuboid (direction of the texture plane) represents the shooting direction, and the thickness of the cuboid (length in the thickness direction with respect to the texture plane) represents the depth of field of the shot image. The size (the size of the texture surface) represents the angle of view in the captured image. The same applies to the icon 33 representing the photographed image of the tree.

  By displaying the icons 31 and 33 as described above on the browser view screen, the user of the digital camera 1 can intuitively and easily grasp the photographing condition when the photographed image is acquired. Note that the information on the shooting conditions as described above is acquired at the time of shooting together with information on the shooting position and shooting time, and is recorded together with the shot image information as part of the above-described meta information.

  On the other hand, like the icon 31, the icon 34 representing the captured image of the mountain has a three-dimensional shape simulating the mountain, and the mountain portion cut out from the captured image is pasted as a texture on the side surface portion. It is attached. Similarly, the icon 32 representing the photographed image of the face has a spherical or disk-shaped three-dimensional shape that simulates a human head, and the side of the icon 32 is the face of the person cut out from the photographed image. Is pasted as a texture.

  In this way, the digital camera 1 displays the portion of the subject cut out in the captured image extracted from the space-time database by pasting it on the solid icon simulating the subject. Thereby, the user of the digital camera 1 can intuitively and easily grasp what is reflected in the captured image of the solid icon. In addition, since the shape of these three-dimensional icons is an example, it can be made into various shapes according to the kind of subject. For example, a human-shaped three-dimensional icon can be used for an entire image of a person, and an animal-shaped three-dimensional icon can be used for an animal. Furthermore, if it is a famous building such as Tokyo Tower, a solid icon having a shape corresponding to the building may be used.

  In the above, the process of cutting out the portion of the subject from the captured image can be automatically performed by the digital camera 1 or the server device 3 using a known processing method such as a main subject recognition process or a face recognition process. Or you may make it perform by the user of the digital camera 1, the administrator of the server apparatus 3, etc. specifying the cutting-out range.

  On the icon 35 representing the character information, a character string “Mt. Fuji” is displayed. If the number of characters is large, not all characters may be displayed, but only some of them may be displayed. On the other hand, the icon 36 representing the voice information is displayed using a specific figure having a balloon shape so that it can be understood that it represents the voice information. At this time, a character, a symbol, or the like may be used in addition to the graphic.

  Note that text information and audio information may be recorded in the space-time database separately from the captured image information, or may be recorded in combination with the captured image information. For example, character information and voice information can be recorded in the space-time database as a part of the above-mentioned meta information added to the captured image information and representing various information. At this time, the photographed image information is extracted from the space-time database, and character information and voice information included in the meta information of the photographed image are extracted together, and an icon as shown in FIG. Each may be displayed on a three-dimensional space. In this case, the contents of the captured image may be automatically analyzed by the digital camera 1 or the server device 3, and the contents of the character information and the voice information to be added as meta information may be determined based on the result. For example, the main subject in the captured image is determined, and character information representing the name of the main subject is recorded in the space-time database as meta information of the captured image. Alternatively, when a user of the digital camera 1 inputs characters or voices during shooting or the like, the contents may be recorded in the space-time database as character information or voice information.

  When character information or audio information is recorded in addition to the captured image information in the space-time database, the type of information to be extracted may be switched according to the change in the attitude of the digital camera 1. For example, if the elevation angle of the digital camera 1 is within a predetermined angle range, the photographed image information is extracted from the space-time database and displayed on the browser view screen. It is extracted from and displayed on the browser view screen. In this way, even if character information or voice information is recorded as meta information added to the photographed image information, the photographed image information and the character information or voice information can be separated and extracted from the space-time database. Therefore, each information can be presented to the user in an easy-to-understand manner.

  When the user changes the tilt of the digital camera 1 while the browser view screen is displayed, the search target time is changed as described above, and different search conditions are set. Therefore, the icon displayed in the pseudo three-dimensional space also changes to a different one. Note that even if the search target time is not set in the search condition, the display target time is set based on the camera tilt so that the icon displayed in the pseudo-stereoscopic space is displayed in the camera as in the case where the search target time is set. You may make it change according to inclination. For example, when the current camera tilt is almost zero, that is, when the user is holding the digital camera 1 facing almost straight, out of the information extracted from the space-time database, a predetermined time range from the present, for example, one year Displays icons of information created or obtained in the previous time range on the browser view screen. When the user tilts the digital camera 1 to the right from this state, the display target time, that is, the range of the acquisition time of information to be displayed as an icon on the browser view screen, is gradually changed back to the past. On the other hand, when the user tilts the digital camera 1 to the left, the display target time is gradually changed from the past to the present.

  When the shutter button 11 is half-pressed, an icon located in the space cursor 30 is selected on the browser view screen at that time, and information on the selected icon is displayed in a list on the monitor 104. An example of the screen displayed at this time is shown in FIG. Hereinafter, such a display screen is referred to as a select view screen. In the select view screen, as the information represented by the icons 31, 32, 35, and 36 located in the space cursor 30 in the browser view screen of FIG. 5, the captured images 37 and 38 corresponding to the icons 31 and 32, respectively, A list of the characters “Mount Fuji” corresponding to the icon 35 and the icon 36 is displayed.

  In the select view screen of FIG. 6, a photographed image 37 surrounded by a frame is selected from the displayed information. The position of this frame can be changed according to the operation of the digital camera 1. For example, the position of the frame may be changed according to a change in posture such as the elevation angle, azimuth angle, and tilt of the digital camera 1. Alternatively, the position of the frame may be changed according to the operation of the jog dial 17. Note that, similarly to the browser view screen, the selection view screen may set the display target time based on the camera tilt to change the acquisition time range of the information to be displayed.

  When the shutter button 11 is fully pressed on the select view screen, the information selected at that time is enlarged and displayed on the monitor 104. The screen displayed at this time is hereinafter referred to as a one-view screen. On the other hand, when the half-press operation of the shutter button 11 is released, the browser view screen of FIG. 5 is restored.

  FIG. 7 shows an example of the one-view screen when the captured image 37 is selected. When the full pressing operation of the shutter button 11 is released, the browser view screen of FIG. 5 is restored. Note that when information other than the photographed image, that is, character information or audio information is selected on the select view screen of FIG. 6, such a one-view screen is not displayed. For example, when character information is selected, the content of the character information is added as a meta information to a captured image that is subsequently captured and recorded. In addition, when audio information is selected, the content of the audio information is output as audio.

  Furthermore, when the user performs a predetermined operation on the digital camera 1, the shooting range when each captured image is acquired is displayed on the monitor 104 by switching from the browser view screen of FIG. 5. FIG. 8 is an example of a shooting range screen displaying a shooting range for each captured image displayed on the browser view screen of FIG. In this shooting range screen, the shooting positions 41 to 44 and the shooting ranges 51 to 54 are displayed in plan view for each of the shot images represented by the icons 31 to 34 in FIG.

  In the shooting range screen of FIG. 8, in the shooting ranges 51 to 54, trapezoidal figures are used to represent the shooting ranges in the respective shot images. For example, in the shooting range 52, line segments 521, 522, and 523 represent the positions of the front depth of field, the focus plane, and the rear depth of field in the shot image, respectively. That is, the farther the distance between the shooting position 42 and the line segment 522 is, the farther the distance from the camera to the main subject is, and the farther the distance between the line segment 521 and the line segment 523 is, the deeper the depth of field is. Represents. The length of the line segment 522 represents the size of the angle of view of the captured image. The same applies to other imaging ranges. Note that, when the focus is adjusted to infinity, the position of the focus plane and the rear depth of field may not be displayed as in the shooting range 54.

  By the way, the browser view screen described above may be displayed overlaid on a background image other than the through image. For example, when a predetermined switching condition is satisfied, the through image is switched and a background image stored in advance by the memory 114 is displayed on the monitor 104. A browser view screen as shown in FIG. 5 is displayed over the background image.

  For the above switching conditions, camera orientation, ambient brightness, and the like can be used. For example, in the current camera direction detected by the motion sensor 108, when the elevation angle exceeds a predetermined threshold, for example, when it is 60 ° or more or −60 ° or less, it is determined that the switching condition is satisfied. Alternatively, the brightness of the surroundings is determined by detecting the brightness of the through image, and when the brightness falls below a predetermined threshold, it is determined that the switching condition is satisfied. In addition to this, various switching conditions can be used.

  In addition, when switching to a through image and displaying a background image, it is preferable to change a background image according to the condition at that time. For example, when the elevation angle is 60 ° or more, an image of the sky or star map corresponding to the camera position at that time is displayed as a background image, and when the elevation angle is −60 ° or more, the camera position at that time A map corresponding to is displayed as a background image. If the ambient brightness is less than a predetermined threshold, a background image simulating the surrounding landscape is displayed according to the camera position at that time. At this time, the time zone of the landscape represented in the background image may be changed according to the camera tilt.

  When the information search mode is set, the digital camera 1 extracts various information from the space-time database and displays it by executing the processing as described above. FIG. 9 is a flowchart showing the flow of processing at this time. This flowchart is executed by the CPU 101. Hereinafter, the flowchart of FIG. 9 will be described.

  In step S10, captured image data based on the imaging signal from the CCD 113 is received from the CCD driver 112, and a through image of the subject is acquired. In step S20, the through image acquired in step S10 is displayed on the monitor 104. Thereby, a through image as shown in FIG. 4 is displayed.

  In step S30, it is determined whether or not the information search mode is set. If the information search mode is set, the process proceeds to the next step S40. On the other hand, if the information search mode is not set and the shooting mode is set, the process returns to step S10. Thereby, a through image as shown in FIG. 4 is displayed in the photographing mode. When the user issues a shooting instruction by operating the shutter button 11, a shot image is acquired according to the shooting instruction, and the shot image information is recorded in the memory card 115.

  In step S40, the camera position is detected. Here, a GPS signal is received by the GPS sensor 107, and a camera position indicating the current position of the digital camera 1 is detected based on the GPS signal. In the next step S50, the camera posture is detected. Here, based on the posture change of the digital camera 1 detected by the motion sensor 108, a camera orientation indicating the current orientation of the digital camera 1 and a camera tilt indicating the current tilt of the digital camera 1 are detected.

  In step S60, based on the camera posture detected in step S50, the information search time range corresponding to the camera tilt at that time is set as described above. In step S70, based on the camera position detected in step S40 and the time range set in step S60, search conditions for searching information from the space-time database are set by the method described above. That is, by setting the search target area based on the camera position and camera orientation, and setting the search target time based on the information search time range set according to the tilt state of the digital camera 1 at that time, Set search conditions. At this time, only the search target area may be set and the search target time may not be set.

  In step S80, information is searched from the space-time database based on the search condition set in step S70. At this time, information may be retrieved from either the space-time database of either the memory card 115 or the server device 3, or information may be retrieved from both space-time databases. In step S90, the information retrieved in step S80 is extracted from the space-time database. Thereby, information satisfying the search condition is extracted.

  When searching the space-time database of the server device 3 in step S80, the search condition set in step S70 may be transmitted to the server device 3 as described above. Alternatively, information recorded in the space-time database may be input to the CPU 101 of the digital camera 1 and the information may be searched for in the digital camera 1. Information retrieved from the space-time database by any method is extracted in step S90.

  In step S100, an icon creation process is performed on the information extracted in step S90. In this icon creation process, icons such as icons 31 to 36 in FIG. 5 are created according to the contents of each information extracted in step S90. Note that information such as shooting conditions necessary at this time is recorded in the space-time database as meta information added to the shot image information as described above.

  In step S110, a three-dimensional process is performed. Here, the range of the pseudo three-dimensional space is determined as described above based on the camera position detected in step S40 and the camera posture detected in step S50.

  In step S120, the browser view screen is displayed on the monitor 104. Here, the grid line and the space cursor corresponding to the range of the pseudo three-dimensional space determined by the three-dimensional processing in step S110 are displayed superimposed on the through image displayed in step S20, and the icon created in step S100 is displayed. Are respectively displayed at corresponding positions in the pseudo three-dimensional space. Thereby, a browser view screen as shown in FIG. 6 is displayed.

  In step S130, it is determined whether or not the camera posture detected by the motion sensor 108 has changed. If there is a change in the camera posture, the process proceeds to step S140. In step S140, the spatial cursor 30 is moved in the browser view screen as described above according to the change in the camera posture. If step S140 is performed, it will progress to step S150. On the other hand, if there is no change in the camera posture, the process proceeds to step S150 without executing step S140.

  In step S150, it is determined whether or not the shutter button 11 is half-pressed. If there is a half-press operation on the browser view screen, the process proceeds to step S160, and if not, the process proceeds to step S210.

  In step S160, it is determined whether there is information in the space cursor 30 or not. If there is information in the space cursor 30, that is, if an icon representing any information is displayed in the space cursor 30 on the browser view screen, the process proceeds to step S170. On the other hand, when there is no information in the space cursor 30, it progresses to step S210.

  In step S170, a select view screen as shown in FIG. 6 is displayed for the information determined to be in the space cursor 30 in step S160. In step S180, it is determined whether or not the shutter button 11 is fully pressed on the select view screen. If there has been a full-press operation, the process proceeds to step S190.

  In step S190, a one-view screen as shown in FIG. 7 is displayed for the photographed image selected on the select view screen when the full-press operation is performed. If character information or audio information other than the shot image has been selected on the select view screen, the contents of the character information can be added to the shot image as meta information instead of displaying the one-view screen as described above. , Perform audio output.

  In step S200, it is determined whether or not the operation of the shutter button 11 has been released. When the operation of the shutter button 11 is released, that is, when the half-press operation or the full-press operation is released, the process returns to step S10. Accordingly, the browser view screen is displayed again by repeating the processing as described above. On the other hand, if the shutter operation is not released, the process returns to step S170 to continue displaying the select view screen. If the full-press operation is performed, the one-view screen is displayed instead of the select-view screen.

  In step S210, it is determined whether or not a background image switching condition is satisfied. If the background image switching condition as described above is satisfied, that is, if the camera orientation or ambient brightness exceeds a predetermined threshold, the process proceeds to step S220. On the other hand, if the switching condition is not satisfied, the process proceeds to step S240.

  In step S220, the background image stored in the memory 114 is read. In step S230, the background image read in step S220 is displayed on the display monitor 104 by switching from the through image displayed in step S20. At this time, as described above, the background image may be changed according to the situation.

  In step S240, it is determined whether or not the user has requested display of the shooting range. If there is a request to display the shooting range by performing a predetermined operation, the process proceeds to step S250. In step S250, each shooting image is acquired by displaying a shooting range screen as shown in FIG. Displays the shooting range. After step S250 is executed, the process returns to step S10. On the other hand, if the predetermined operation is not performed and there is no request for displaying the photographing range, the process returns to step S10 without executing step S250.

  As described above, the digital camera 1 detects the camera position indicating the current position of the digital camera 1 based on the GPS signal received by the GPS sensor 107 (step S40). Then, a search condition is set based on the detected camera position (step S70), and any information is searched from the space-time database based on the set search condition (step S80) and extracted (step S90). . The information thus extracted is displayed on the monitor 104 on the browser view screen (step S120). Therefore, it is possible to easily search and display desired information without inputting a search keyword.

According to the embodiment described above, the following operational effects are obtained.
(1) The digital camera 1 inputs information having position-related data as meta information from the space-time database by the processing of the CPU 101. Further, the current position of the digital camera 1, that is, the current position of the CCD 113 that captures the subject is detected (step S40), and the input information is searched based on the detected position (step S80). Then, the information retrieved in step S80 is displayed together with the image captured by the CCD 113 on the monitor 104 according to the data related to the position of the information (step S120). Since it did in this way, even if it does not input the keyword for a search, desired information can be searched and displayed easily.

(2) The monitor 104 arranges and displays the information searched in step S80 at a position in the image corresponding to the position data of the meta information included in the information. Since it did in this way, the positional relationship of the searched information and an image can be displayed correctly, and the information which is easy to understand can be displayed.

(3) The information input and searched from the space-time database includes information that is an image captured by the digital camera 1. That is, the CPU 101 searches for an image picked up by the CCD 113 having information on the position detected by the CPU 101. Since it did in this way, even if it does not input the keyword for a search, the image imaged in the past can be searched and displayed easily.

(4) The digital camera 1 detects the orientation of the digital camera 1 based on the posture change of the digital camera 1 detected by the motion sensor 108 by the processing of the CPU 101 (step S50). In step S80, information is searched based on the position detected in step S40 by the CPU 101 and the direction detected in step S50. That is, in step S50, the direction of the digital camera 1 is detected in at least one of the horizontal direction and the vertical direction, and information is searched by setting a search condition based on the detection result and the position detection result in step S70. To do. Since it did in this way, compared with the case where information is searched based only on the detection result of a position, the information search which reflected the present condition more accurately can be performed.

(5) The digital camera 1 further detects the tilt of the digital camera 1 based on the posture change of the digital camera 1 detected by the motion sensor 108 by the processing of the CPU 101 (step S50). Depending on this inclination, the time range in which the information to be searched for in step S80 is created can be changed. In this way, it is possible to search for information by setting a search condition according to time in addition to the place.

(6) The digital camera 1 acquires a through image captured by the CCD 113 (step S10). When the browser view screen is displayed on the monitor 104 in step S120, the retrieved information is displayed on the through image. Since it did in this way, the user can see the information inputted and searched from the space-time database while observing the state of the subject with the through image.

(7) When a predetermined switching condition is satisfied, the background image stored in advance by the memory 114 is switched from the through image and displayed on the monitor 104 (step S230). Since it did in this way, the searched information can be displayed easily.

(8) The switching condition for switching from the through image to the background image can be set according to the camera orientation or the ambient brightness. Therefore, when an object that should be the main subject is not displayed in the through image or when the main subject is not well captured in the through image because the surroundings are dark, an appropriate background image is used instead of the through image on the monitor 104. Can be displayed.

(9) In the space-time database, it is possible to record at least one of photographed images, characters, and sounds as information. In step S80, the digital camera 1 can search for any one of the photographed image, text, and voice. Since it did in this way, not only a picked-up image but various information can be searched from a space-time database, and can be shown to a user.

(10) The digital camera 1 further displays the shooting range of the shot image searched in step S80 on the monitor 104 when the user requests to display the shooting range by performing a predetermined operation ( Step S250). Since it did in this way, the user can know easily how imaging was performed.

(11) Information in the space-time database searched by the digital camera 1 is stored in the memory card 115 loaded in the digital camera 1 or the server device 3 connected to the digital camera by wireless communication. Since it did in this way, in addition to the picked-up image acquired with the digital camera 1, various information, such as a picked-up image by another camera, can also be made into a search object.

  In the above embodiment, the digital camera 1 which is an electronic camera has been described as an example. However, the present invention can also be applied to other devices. For example, the present invention can be applied to a portable electronic device such as a mobile phone with a camera. Alternatively, the present invention may be applied when searching and displaying various types of information such as image information recorded in a space-time database in advance in digital binoculars or field scopes that do not perform shooting. That is, an input unit that inputs information having data related to a position, an image input unit that inputs an image of a subject, a position detection unit that detects a position, and information search based on the position detected by the position detection unit The present invention can also be applied to a display device that includes a search unit that performs a search, and a display unit that displays information searched by the search unit together with an image according to data relating to the position of the information.

  The embodiment and various modifications described above are merely examples. Therefore, the present invention is not limited to these contents as long as the characteristics of the present invention are not impaired.

It is a block diagram of the camera system by one embodiment of this invention. It is a figure which shows the external appearance of the digital camera by one embodiment of this invention. 1 is a block diagram of a digital camera according to an embodiment of the present invention. It is a figure which shows the example of the through image displayed when imaging | photography mode is set. It is a figure which shows the example of the browser view screen displayed when information search mode is set. It is a figure which shows the example of a select view screen. It is a figure which shows the example of a one view screen. It is a figure which shows the example of an imaging | photography range screen. It is a figure which shows the flowchart which shows the flow of a process when extracting and displaying various information from a space-time database.

Explanation of symbols

1: Digital camera 2: Access point 3: Server device

Claims (19)

An input unit for inputting information having data on the position;
An imaging unit for imaging a subject;
A position detector for detecting the position;
A search unit for searching for the information based on the position detected by the position detection unit;
A display unit that displays the information searched by the search unit together with an image captured by the imaging unit in accordance with data related to a position of the information ;
An inclination detection unit for detecting the inclination of the electronic camera;
The information includes time data when the information is created,
The search unit changes a time range in which the information to be searched is created according to the tilt detected by the tilt detection unit.
An electronic camera characterized by The electronic camera according to claim 1,
The image is a through image,
The display unit displays a background image stored in advance by switching from the through image when a predetermined switching condition is satisfied.
An electronic camera characterized by The electronic camera according to claim 2,
The switching condition is set according to the orientation of the electronic camera or the ambient brightness.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 3,
The display unit displays a line indicating a pseudo three-dimensional space together with the image.
An electronic camera characterized by An input unit for inputting information having data on the position;
An imaging unit for imaging a subject;
A position detector for detecting the position;
A search unit for searching for the information based on the position detected by the position detection unit;
A display unit that displays the information searched by the search unit together with an image captured by the imaging unit in accordance with data relating to the position of the information;
The image is a through image,
The display unit displays a background image stored in advance by switching from the through image when a predetermined switching condition is satisfied.
An electronic camera characterized by The electronic camera according to claim 5,
The switching condition is set according to the orientation of the electronic camera or the ambient brightness.
An electronic camera characterized by The electronic camera according to claim 5 or 6,
The display unit displays a line indicating a pseudo three-dimensional space together with the image.
An electronic camera characterized by An input unit for inputting information having data on the position;
An imaging unit for imaging a subject;
A position detector for detecting the position;
A search unit for searching for the information based on the position detected by the position detection unit;
A display unit that displays the information searched by the search unit together with an image captured by the imaging unit in accordance with data relating to the position of the information;
The display unit displays a line indicating a pseudo three-dimensional space together with the image.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 8,
The display unit arranges the information at a position in the image corresponding to position data included in the information.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 9,
The information is an image picked up by the image pickup unit having information on the position detected by the position detection unit.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 10,
An orientation detection unit for detecting the orientation of the electronic camera;
The search unit searches for the information based on a position detected by the position detection unit and a direction detected by the direction detection unit.
An electronic camera characterized by The electronic camera according to claim 11,
The orientation detection unit detects at least one of a horizontal direction and a vertical direction,
The search unit searches for the information based on a position detected by the position detection unit and a direction detected by the direction detection unit.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 12,
The information is at least one of a photographed image, text, and voice.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 13,
The display unit further displays a shooting range of the shot image searched by the search unit.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 14,
The information is stored in a recording medium loaded in the electronic camera or a server device connected to the electronic camera by wireless communication.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 15,
A selection unit for selecting the information;
The selection unit includes at least one of a zoom button, a rotatable operation member, and a cross key.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 15,
A selection unit for selecting the information;
When the information is selected by the selection unit and a button for instructing photographing is operated, information related to the information selected by the selection unit is displayed.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 17,
The display unit displays at least one of a field angle and a depth of field of the captured image searched by the search unit.
An electronic camera characterized by The electronic camera according to any one of claims 1 to 18,
Capable of setting a shooting mode for acquiring a shot image and a search mode for searching for the information when shooting is instructed
An electronic camera characterized by

Images