JP6221465B2 - Information processing apparatus and program - Google Patents

Description

  The present invention relates to an information processing apparatus and a program.

  A mobile terminal such as a mobile phone or a smartphone is equipped with a camera, and can be easily photographed using the mobile terminal. At event venues and sightseeing spots, many people take commemorative photos using mobile devices instead of digital still cameras. In addition, a camera of a mobile terminal may be used for reading a QR code (registered trademark), a video phone, or the like. In addition, various application programs have been developed to more effectively use the camera of the mobile terminal.

  For example, a first method is proposed in which the movement of the mobile terminal in a direction perpendicular to the display screen is detected using a three-axis acceleration sensor, and the display size of the image captured by the camera is changed according to the amount of movement. Has been. According to the first method, the display size of the image can be changed with a natural feeling as if the “magnifying glass” is moving. In addition, a second method has been proposed in which position data is acquired from an image of a printed material captured by a camera, and a multimedia content in which an image prepared in advance and an image of the printed material are aligned and displayed is displayed.

Japanese Patent Laying-Open No. 2005-25170 JP 2002-259991 A

  There are many opportunities to carry mobile devices. Therefore, if the first method is applied, desired information can be easily obtained by using a portable terminal close to the user when reading a printed matter in which fine characters are written or when checking details of a map. It is considered possible. However, in the method of displaying an image captured by the camera as in the first method, a clear image cannot be obtained if the mobile terminal is too close to the object to be imaged. For example, when the mobile terminal is closer to the shooting target than the shortest shooting distance of the lens, the focus is not achieved. Furthermore, when the object to be photographed and the shadow of the portable terminal overlap, the exposure amount is insufficient.

  Moreover, if the image of the imaging target prepared in advance is displayed in alignment with the image captured by the camera as in the second method, a suitable image is displayed regardless of the distance between the mobile terminal and the imaging target. be able to. However, the alignment processing as in the second method has a high calculation load. Therefore, if the alignment process is executed each time the mobile terminal moves, it may cause other processes to be delayed or increase battery consumption. Therefore, according to one aspect, an object of the present invention is to provide an information processing apparatus and a program that can reduce the load of display processing.

  According to one aspect of the present disclosure, an image capturing unit that captures an object, a storage unit that stores an image, an image of the object captured by the image capturing unit is displayed on a screen in real time, and at least one sheet is displayed. The image is stored in the storage unit, and when the set condition is satisfied, the image stored in the storage unit is read, and the partial image corresponding to the current imaging range is extracted from the read image, and the screen display is partially An information processing apparatus having a control unit that switches to an image is provided.

  According to another aspect of the present disclosure, an image of an object captured by a camera is displayed on a screen in real time on a processor of a computer having a camera, a memory, and a processor, and at least one image is displayed. Is stored in the memory, and when the set condition is satisfied, the image stored in the memory is read, the partial image corresponding to the current imaging range is extracted from the read image, and the screen display is switched to the partial image. A program for executing processing is provided.

  As described above, according to the present invention, it is possible to reduce the load of display processing.

It is the figure which showed an example of the information processing apparatus which concerns on 1st Embodiment. It is the 1st figure showing an example of the display method by the information processor concerning a 2nd embodiment. It is the 2nd figure showing an example of the display method by the information processor concerning a 2nd embodiment. It is the figure which showed an example of the hardware which can implement | achieve the function which the information processing apparatus which concerns on 2nd Embodiment has. It is a 1st block diagram for demonstrating the function of the information processing apparatus which concerns on 2nd Embodiment. It is a 2nd block diagram for demonstrating the function of the information processing apparatus which concerns on 2nd Embodiment. It is a 3rd block diagram for demonstrating the function of the information processing apparatus which concerns on 2nd Embodiment. It is a figure for demonstrating the image holding function which the information processing apparatus which concerns on 2nd Embodiment has. It is a figure for demonstrating the switching determination function (stationary determination) which the information processing apparatus which concerns on 2nd Embodiment has. It is a figure for demonstrating the area | region calculation function which the information processing apparatus which concerns on 2nd Embodiment has. It is the flowchart which showed the flow of the display process which the information processing apparatus which concerns on 2nd Embodiment performs. It is the flowchart which showed the flow of the switching determination process which the information processing apparatus which concerns on 2nd Embodiment performs. It is a 1st figure for demonstrating the switching determination function which the information processing apparatus which concerns on the modification (modification # 1) of 2nd Embodiment has. It is a 2nd figure for demonstrating the switching determination function which the information processing apparatus which concerns on the modification (modification # 1) of 2nd Embodiment has. It is the flowchart which showed the flow of the switching determination process which the information processing apparatus which concerns on the modification (modification # 1) of 2nd Embodiment performs. It is a figure for demonstrating the switching determination function which the information processing apparatus which concerns on the modification (modification # 2) of 2nd Embodiment has. It is the flowchart which showed the flow of the switching determination process which the information processing apparatus which concerns on the modification (modification # 2) of 2nd Embodiment performs. It is a figure for demonstrating the switching determination function which the information processing apparatus which concerns on the modification (modification # 3) of 2nd Embodiment has. It is the flowchart which showed the flow of the switching determination process which the information processing apparatus which concerns on the modification (modification # 3) of 2nd Embodiment performs. It is the flowchart which showed the flow of the switching determination process which the information processing apparatus which concerns on the modification (modification # 4) of 2nd Embodiment performs. It is a 1st figure for demonstrating the recording function of the captured image which the information processing apparatus which concerns on the modification (modification # 5) of 2nd Embodiment has. It is a 2nd figure for demonstrating the recording function of the captured image which the information processing apparatus which concerns on the modification (modification # 5) of 2nd Embodiment has. It is a 3rd figure for demonstrating the recording function of the captured image which the information processing apparatus which concerns on the modification (modification # 5) of 2nd Embodiment has. It is the flowchart which showed the flow of the display process which the information processing apparatus which concerns on the modification (modification # 5) of 2nd Embodiment performs. It is the flowchart which showed the flow of the recording process of the captured image which the information processing apparatus which concerns on the modification (modification # 5) of 2nd Embodiment performs. It is a 1st figure for demonstrating the recording function of the captured image which the information processing apparatus which concerns on the modification (modification # 6) of 2nd Embodiment has. It is a 2nd figure for demonstrating the recording function of the captured image which the information processing apparatus which concerns on the modification (modification # 6) of 2nd Embodiment has. It is the flowchart which showed the flow of the recording process of the captured image which the information processing apparatus which concerns on the modification (modification # 6) of 2nd Embodiment performs.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. In addition, about the element which has the substantially same function in this specification and drawing, duplication description may be abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. First Embodiment>
The first embodiment will be described with reference to FIG. FIG. 1 is a diagram illustrating an example of an information processing apparatus according to the first embodiment.

As illustrated in FIG. 1, the information processing apparatus 10 includes an imaging unit 11, a storage unit 12, a display unit 13, and a control unit 14.
The imaging unit 11 includes, for example, an optical system such as a lens, an imaging element such as a charge coupled device (CCD) and a complementary metal-oxide semiconductor (CMOS), an analog-to-digital converter (AD converter), and signal processing. A device that includes a circuit and the like.

The storage unit 12 is, for example, a volatile storage device such as a RAM (Random Access Memory) or a nonvolatile storage device such as an HDD (Hard Disk Drive) or a flash memory.
The display unit 13 is a display device such as CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), PDP (Plasma Display Panel), or ELD (Electro-Luminescence Display).

  The control unit 14 is, for example, a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The control unit 14 may be an electronic circuit other than a processor such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). For example, the control unit 14 executes a program stored in the storage unit 12 or another memory.

  The imaging unit 11 images the object 21. Examples of the object 21 include a printed matter and a map on which characters and photographs are posted. The storage unit 12 stores an image. However, the storage unit 12 may store information other than images. The display unit 13 has a screen on which an image is displayed. For example, the image captured by the imaging unit 11 is displayed on the screen of the display unit 13. The image stored in the storage unit 12 is displayed on the screen of the display unit 13.

  The control unit 14 displays the image of the object 21 captured by the imaging unit 11 on the screen in real time. Further, the control unit 14 causes the storage unit 12 to store at least one image 23b. For example, the control unit 14 causes the storage unit 12 to store the image 23b according to the imaging operation performed by the user. Note that the timing of storing the image 23b in the storage unit 12 may be automatically determined according to the acceleration change of the information processing apparatus 10, or may be automatically determined based on the relationship between a plurality of captured images. Good.

  In the example of FIG. 1, the information processing apparatus 10 moves from the position A to the position B, and the distance between the information processing apparatus 10 and the target object 21 is D2 (D2 <D1). When the information processing apparatus 10 approaches the object 21, the shadow 24 of the information processing apparatus 10 overlaps the object 21, and the image 23a captured by the imaging unit 11 becomes dark. If the distance D2 is sufficiently short (for example, shorter than the shortest shooting distance), the image cannot be focused and the image 23a captured by the imaging unit 11 is blurred.

  When the set condition is satisfied, the control unit 14 reads the image 23b stored in the storage unit 12. For example, the above condition is set when the image picked up by the image pickup unit 11 becomes dark or when the image is out of focus. As a method for determining whether or not the set condition is satisfied, for example, a method of using a change in an image captured by the imaging unit 11 is conceivable. When an acceleration sensor is mounted on the information processing apparatus 10, the above condition is set when the acceleration change of the information processing apparatus 10 continues to be below a set threshold for a set time. You can also.

  The control unit 14 extracts a part of the image corresponding to the current imaging range 22 (hereinafter referred to as a partial image) from the read image 23b, and switches the screen display to the partial image. In the example of FIG. 1, when the distance between the information processing apparatus 10 and the target 21 is D2, the image 23a currently captured has insufficient brightness or blur. However, since the partial image of the read image 23b is displayed on the screen, a suitable image with insufficient lightness or blur is displayed on the display unit 13.

  As in the example illustrated in FIG. 1, after the display on the screen is switched to the partial image, the display unit 13 even in a state where the information processing device 10 and the object 21 are in contact (the information processing device 10 is in the position C). Is displayed with a suitable image.

  As described above, before the set condition is satisfied, the image captured by the imaging unit 11 is displayed on the display unit 13, so that a suitable image display can be realized without performing image alignment processing. . On the other hand, after satisfying the set condition, a partial image of the image stored in the storage unit 12 is displayed on the display unit 13, so that a suitable image display is maintained. Therefore, according to the information processing apparatus 10, a suitable image display can be obtained regardless of the shooting distance. In addition, since the partial image is not used in a state before the set condition is satisfied, it is possible to suppress the calculation load by an amount that does not require at least partial image alignment processing.

The first embodiment has been described above.
<2. Second Embodiment>
Next, a second embodiment will be described.

[2-1. Display method]
First, a display method by the information processing apparatus 100 according to the second embodiment will be described with reference to FIGS. 2 and 3. FIG. 2 is a first diagram illustrating an example of a display method performed by the information processing apparatus according to the second embodiment. FIG. 3 is a second diagram illustrating an example of a display method performed by the information processing apparatus according to the second embodiment.

  As illustrated in FIG. 2, the information processing apparatus 100 has a function of capturing an image of the object M and displaying the captured image on a screen. For example, when the information processing apparatus 100 is at the position A1, the information processing apparatus 100 displays a captured image of the object M captured at the position A1 on the screen as if the object M is seen through the casing. Hereinafter, such a display method may be referred to as transmissive display. The transmissive display can be realized by displaying the captured image captured by the information processing apparatus 100 on the screen in real time. For example, the information processing apparatus 100 moved to the position A2 displays the captured image captured at the position A2 on the screen in real time.

  In addition, when the user performs an enlargement operation or a reduction operation, the information processing apparatus 100 changes the display size of the captured image displayed on the screen according to the operation content. For example, the information processing apparatus 100 enlarges the captured image by image processing and displays a part of the enlarged captured image on the screen. When the zoom lens is mounted on the information processing apparatus 100, a mechanism may be adopted in which a part of a captured image captured by driving the focal length of the zoom lens to the tele end side is displayed on the screen. it can. However, in the following description, for the sake of simplicity, an information processing apparatus 100 equipped with a single focus lens is considered. In addition, a part of the captured image may be referred to as a partial image.

  As a method for realizing the transmissive display, in addition to the method of displaying the captured image as it is on the screen as described above, one image of the object M is prepared, and the position of the information processing apparatus 100 is determined from the image. A method of cutting out the imaging range of the target M and displaying it on the screen is conceivable. For example, the information processing apparatus 100 at the position A1 calculates the imaging range at the position A1, and selects an area corresponding to the imaging range (hereinafter, referred to as an image of the object M prepared in advance (hereinafter referred to as a retained image)). Target area). The detection of the target area can be realized, for example, by performing pattern matching between the actual captured image and the retained image.

  At position A1, the information processing apparatus 100 cuts out the target area from the retained image and displays it on the screen. When the information processing apparatus 100 moves from the position A1 to the position A2, the information processing apparatus 100 calculates a target area at the position A2, cuts out the calculated target area from the retained image, and displays it on the screen. That is, in this method, calculation of the target region by pattern matching occurs every time the position of the information processing apparatus 100 changes. Similar calculations occur when the information processing apparatus 100 is tilted with respect to the XY plane, as well as when the information processing apparatus 100 moves in the XY plane parallel to the object M.

  The greater the distance between the object M and the information processing apparatus 100, the greater the influence that changes in posture and position of the information processing apparatus 100 have on the appearance of the image. Will increase. In order to suppress such an increase in calculation load, the information processing apparatus 100 switches the transmissive display mechanism in accordance with the positional relationship between the object M and the information processing apparatus 100, as shown in FIG.

  FIG. 3 illustrates a state in which the information processing apparatus 100 moves in the order of positions B1, B2, and B3. The distance D1 between the information processing apparatus 100 at the position B1 and the object M is large. A distance D2 between the information processing apparatus 100 at the position B2 and the object M is smaller than D1. The information processing apparatus 100 at the position B3 is placed on the object M. That is, in the example of FIG. 3, the information processing apparatus 100 is shown moving in the direction approaching the object M (Z direction).

  As described above, the distance between the object M and the information processing apparatus 100 affects the calculation load applied when calculating the target area. Therefore, when the distance between the object M and the information processing apparatus 100 is larger than a certain distance, the information processing apparatus 100 is low by a method of displaying a captured image on the screen in real time (hereinafter, real time display). Realize transparent display with computational load. However, in the case of real-time display, if the distance between the object M and the information processing apparatus 100 is smaller than a certain distance, a clear captured image may not be obtained.

  For example, since the shortest shooting distance (shortest shooting distance) at which the subject can be focused is determined for each lens, the object M and the information processing apparatus 100 (strictly speaking, an image sensor) than the shortest shooting distance of the lens. If the distance between is too short, it will not be in focus. If the information processing apparatus 100 is too close to the object M, the shadow of the information processing apparatus 100 overlaps the object M and the image becomes dark. This state is shown in FIG. 3 (when the information processing apparatus 100 is located at the position B2). The captured image captured by the information processing apparatus 100 at the position B2 is an unclear image that is dark and out of focus.

  Therefore, the information processing apparatus 100 stores a clear captured image in advance, and uses the retained image (hereinafter, retained image display) before the captured image becomes unclear according to the position change of the information processing apparatus 100. ) To switch the display mechanism to transparent. When switching to the holding image display, as described above, the calculation load for the calculation of the target region is generated, but since the distance between the information processing apparatus 100 and the object M is small, transparent display can be realized with a relatively small calculation load. Can do. Further, even when the information processing apparatus 100 is moved to the position B3 (when the information processing apparatus 100 is in close contact with the object M), the contents of the object M can be displayed.

  The technique according to the second embodiment can be applied to various application programs. As an example, a case where the technique is applied to a magnifying glass application program will be considered. When the information processing apparatus 100 is brought close to the object M, the application program is displayed in an enlarged manner as if the object M is being viewed with a magnifying glass. Note that the enlargement magnification is set in advance. When the user brings the information processing apparatus 100 close to the object M, a part of the object M is enlarged and displayed on the screen of the information processing apparatus 100 according to the distance between the information processing apparatus 100 and the object M, and a magnifying glass. It looks the same as looking at the object M.

  When the transparent display is realized using only the real-time display, if the user brings the information processing apparatus 100 too close to the object M, the display is disturbed or the display is completely dark. On the other hand, when the transmissive display is realized by using only the retained image display, there is a possibility that the display may be delayed depending on the performance of the processor, and the heat generation amount and the battery consumption amount may be increased.

  However, according to the information processing apparatus 100 described above, transmissive display is possible even when the information processing apparatus 100 is in close contact with the object M, and display delay, increase in heat generation, increase in battery consumption, and the like are unlikely to occur. . Therefore, it can be said that the magnifying glass application program is a suitable application example of the technique according to the second embodiment. Of course, the technology according to the second embodiment can be applied to any application program that uses transparent display.

The display method by the information processing apparatus 100 has been described above.
[2-2. hardware]
Next, the hardware of the information processing apparatus 100 will be described with reference to FIG. FIG. 4 is a diagram illustrating an example of hardware capable of realizing the functions of the information processing apparatus according to the second embodiment.

  The functions of the information processing apparatus 100 can be realized using, for example, hardware resources shown in FIG. That is, the functions of the information processing apparatus 100 are realized by controlling the hardware shown in FIG. 4 using a computer program.

  As shown in FIG. 4, this hardware mainly includes a CPU 902, a ROM (Read Only Memory) 904, a RAM 906, a host bus 908, and a bridge 910. The hardware further includes an external bus 912, an interface 914, an input unit 916, an output unit 918, a storage unit 920, a drive 922, a connection port 924, and a communication unit 926.

  The CPU 902 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation of each component or a part thereof based on various programs recorded in the ROM 904, the RAM 906, the storage unit 920, or the removable recording medium 928. . The ROM 904 is an example of a storage device that stores a program read by the CPU 902, data used for calculation, and the like. The RAM 906 temporarily or permanently stores, for example, a program read by the CPU 902 and various parameters that change when the program is executed.

  These elements are connected to each other via, for example, a host bus 908 capable of high-speed data transmission. On the other hand, the host bus 908 is connected to an external bus 912 having a relatively low data transmission speed via a bridge 910, for example. As the input unit 916, for example, a mouse, a keyboard, a touch panel, a touch pad, a button, a switch, a lever, or the like is used. Furthermore, as the input unit 916, a remote controller capable of transmitting a control signal using infrared rays or other radio waves may be used.

  As the output unit 918, for example, a display device such as a CRT, LCD, PDP, or ELD is used. As the output unit 918, an audio output device such as a speaker or headphones, or a printer may be used. In other words, the output unit 918 is a device that can output information visually or audibly.

  The storage unit 920 is a device for storing various data. As the storage unit 920, for example, a magnetic storage device such as an HDD is used. Further, as the storage unit 920, a semiconductor storage device such as an SSD (Solid State Drive) or a RAM disk, an optical storage device, a magneto-optical storage device, or the like may be used.

  The drive 922 is a device that reads information recorded on a removable recording medium 928 that is a removable recording medium or writes information on the removable recording medium 928. As the removable recording medium 928, for example, a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is used.

  The connection port 924 is a port for connecting an external connection device 930 such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal. For example, a camera, a microphone, an earphone, an infrared light source, or the like is used as the external connection device 930.

  The communication unit 926 is a communication device for connecting to the network 932. As the communication unit 926, for example, a communication circuit for wired or wireless LAN (Local Area Network), a communication circuit for WUSB (Wireless USB), a communication circuit or router for optical communication, an ADSL (Asymmetric Digital Subscriber Line) Communication circuits, routers, communication circuits for mobile phone networks, and the like are used. A network 932 connected to the communication unit 926 is a wired or wireless network, and includes, for example, the Internet, a LAN, a broadcast network, a satellite communication line, and the like.

The hardware of the information processing apparatus 100 has been described above.
[2-3. function]
Next, functions of the information processing apparatus 100 will be described with reference to FIGS. FIG. 5 is a first block diagram for explaining functions of the information processing apparatus according to the second embodiment. FIG. 6 is a second block diagram for explaining functions of the information processing apparatus according to the second embodiment. FIG. 7 is a third block diagram for explaining functions of the information processing apparatus according to the second embodiment. In the description, FIGS. 8 to 10 will be referred to as appropriate.

  As illustrated in FIG. 5, the information processing apparatus 100 includes a position information acquisition unit 101, an imaging unit 102, a control unit 103, a display unit 104, and a storage unit 105. Note that the functions of the position information acquisition unit 101 and the control unit 103 can be realized using, for example, the CPU 902 described above. The function of the display unit 104 can be realized by using, for example, the output unit 918 described above. The function of the storage unit 105 can be realized using, for example, the RAM 906 and the storage unit 920 described above.

(Position information acquisition unit 101)
The position information acquisition unit 101 is an element that acquires information related to the position of the information processing apparatus 100. For example, the position information acquisition unit 101 includes an acceleration sensor 111 and a movement information calculation unit 112 as shown in FIG.

  The acceleration sensor 111 is, for example, a three-axis acceleration sensor that detects an acceleration value ax in the X-axis direction, an acceleration value ay in the Y-axis direction, and an acceleration value az in the Z-axis direction. As an applicable triaxial acceleration sensor, there is a semiconductor type acceleration sensor using MEMS (Micro Electro Mechanical Systems). For example, a capacitance type acceleration sensor that detects acceleration from a change in capacitance, a piezoresistive type acceleration sensor that detects acceleration using a piezoresistive element, or the like can be used. Of course, a triaxial acceleration sensor according to another method is also applicable.

  The acceleration value for each axis output from the acceleration sensor 111 is input to the movement information calculation unit 112 and the control unit 103. The movement information calculation unit 112 integrates the acceleration value for each axis detected by the acceleration sensor 111 with respect to time, and calculates the movement direction and movement amount of the information processing apparatus 100. Information indicating the direction and amount of movement of the information processing apparatus 100 calculated by the movement information calculation unit 112 (hereinafter referred to as movement information) is input to the control unit 103.

(Imaging unit 102)
The imaging unit 102 is an element that images the object M. An image (captured image) of the object M captured by the imaging unit 102 is input to the control unit 103. Note that while the application program using the transparent display is operating, the imaging unit 102 continuously captures the object M, and sequentially inputs the captured images to the control unit 103. In addition, you may make it the structure in which a user can operate the time which continues taking a captured image, and the timing of the start and end of imaging. In this case, a captured image of the object M captured in response to a user operation is input to the control unit 103.

(Control unit 103)
The control unit 103 is an element that performs recording of a captured image (selection of a retained image), calculation of a target area, display of an image, switching from real-time display to retained image display, switching from retained image display to real-time display, and the like. is there.

As illustrated in FIG. 7, the control unit 103 includes a display control unit 131, an operation information acquisition unit 132, an image recording unit 133, a switching determination unit 134, and a target area calculation unit 135.
The display control unit 131 displays the captured image or the retained image on the display unit 104. The operation information acquisition unit 132 acquires operation information input by the user. For example, when the user performs a shutter operation, the operation information acquisition unit 132 acquires operation information indicating the shutter operation. When the user performs an operation for enlarging the display size, the operation information acquisition unit 132 acquires operation information indicating the enlargement operation. When the user performs a display size reduction operation, the operation information acquisition unit 132 acquires operation information indicating the reduction operation.

  Operation information indicating the shutter operation is input to the image recording unit 133. In addition, operation information indicating an enlargement operation is input to the target area calculation unit 135 together with magnification information. In addition, operation information indicating a reduction operation is input to the target area calculation unit 135 together with magnification information.

  When the operation information indicating the shutter operation is input, the image recording unit 133 records the captured image captured by the imaging unit 102 in the storage unit 105 as illustrated in FIG. FIG. 8 is a diagram for explaining the image holding function of the information processing apparatus according to the second embodiment. Note that the captured image recorded in the storage unit 105 by the image recording unit 133 is referred to as a retained image. The width of the retained image is denoted by W, the height of the retained image is denoted by H, and the central coordinates of the retained image are denoted by O. The position of the information processing apparatus 100 at the time of capturing the retained image is referred to as a reference position.

  The switching determination unit 134 determines whether or not the set condition is satisfied. When the set condition is satisfied, the switching determination unit 134 switches the display method from real-time display to retained image display. When the set condition is not satisfied, the switching determination unit 134 switches the display method from the retained image display to the real-time display.

  Several setting methods of conditions can be considered. Here, an example in which the display method is switched according to the acceleration change of the information processing apparatus 100 in the Z-axis direction will be described with reference to FIG. Other examples (modifications) will be described later. FIG. 9 is a diagram for describing a switching determination function (stillness determination) included in the information processing apparatus according to the second embodiment.

  The switching determination unit 134 records the acceleration value az in the Z-axis direction input from the position information acquisition unit 101 for each time Tz. At this time, the switching determination unit 134 records the acceleration value az from the current time to the time Ta.

  Note that the times Tz and Ta are set in advance. Also, the acceleration value at the current time is expressed as az [n] (n = Ta / Tz), the acceleration value at the time before time Ta is expressed as az [1], and recorded from the time before time Ta to the current time. The acceleration values are expressed in order as az [j] (j = 1,..., N). A ring buffer or the like may be used for recording.

  FIG. 9 shows changes in the acceleration value az during a certain period. If the change in the acceleration value az at time Ta is 0 or a value close to 0, the switching determination unit 134 switches from real-time display to retained image display.

  For example, the switching determination unit 134 compares the change amount azz of the acceleration value az in the section dt1 with a preset value azz_max close to 0. Note that the change amount azz is smaller than the value azz_max in the section dt1. In this case, the switching determination unit 134 switches the display method from real-time display to retained image display.

  Further, the switching determination unit 134 compares the change amount azz of the acceleration value az in the section dt2 with a preset value azz_max close to 0. Note that the amount of change azz is smaller than the value azz_max in the section dt2. In this case, the switching determination unit 134 maintains the retained image display without switching the display method.

  Further, the switching determination unit 134 compares the change amount azz of the acceleration value az in the section dt3 with a preset value azz_max close to 0. Note that the change amount azz is larger than the value azz_max in the section dt3. In this case, the switching determination unit 134 switches the display method from the retained image display to the real-time display.

  By the way, when viewing a newspaper or a map using a magnifying glass, a person moves the magnifying glass up and down until characters and figures are displayed in a desired size. When characters and figures are displayed in a desired size, a person starts reading a newspaper or a map without moving the magnifying glass up and down from the current position. While reading, the magnifying glass may be moved in a direction parallel to the newspaper or map to change the reading range, but the magnifying glass is not suddenly moved toward or away from the newspaper or map. Such a change in behavior can be detected based on a change in acceleration in the Z-axis direction.

  It is reasonable to display a clear image by holding image display while observing the object M, and to reduce the computational load by displaying in real time when a person adjusts the size of characters and figures. is there. That is, a good balance between display quality and calculation load can be realized. In addition, if the value azz_max is adjusted to 0, when the information processing apparatus 100 is placed on the object M, it can be transformed into a mechanism for switching from real-time display to retained image display.

  Information indicating whether the display is real-time display or retained image display is input to the display control unit 131 and the target area calculation unit 135. In the case of holding image display, the target area calculation unit 135 calculates a target area corresponding to the imaging range of the information processing apparatus 100 at the current time, as shown in FIG. FIG. 10 is a diagram for explaining the area calculation function of the information processing apparatus according to the second embodiment.

  It is assumed that the information processing apparatus 100 is at the position P. The target area calculation unit 135 uses the position information of the reference position and the position P input from the position information acquisition unit 101 and the magnification information (hereinafter, magnification B) input from the operation information acquisition unit 132.

The target area calculation unit 135 calculates the reference magnification A according to the following equation (1) based on the viewing angle θ (see FIG. 10B), the Z coordinate Z _P of the position P, and the width W of the retained image. In addition, the target area calculation unit 135 calculates the display magnification C defined by the following equation (2). Then, the target area calculation unit 135 calculates the width C * W and the height C * H of the target area based on the display magnification C. In addition, the target area calculation unit 135 calculates the center coordinates (X _P , Y _P ) of the target area based on the moving direction and the moving amount from the reference position to the position P (see reference sign V in FIG. 10C). To do.

The width C * W and height C * H, the center coordinates (X _P , Y _P ), and the display magnification C of the target area calculated by the target area calculation unit 135 are input to the display control unit 131. Then, the display control unit 131 cuts out a partial image corresponding to the target area from the retained image, and displays an image obtained by multiplying the partial image by 1 / C on the display unit 104.

(Display unit 104, storage unit 105)
The display unit 104 displays a captured image or a retained image. The storage unit 105 stores the retained image. In addition, it can also be modified to a mechanism for displaying a captured image or a retained image on a display device connected to the outside of the information processing apparatus 100. In addition, the storage device connected to the outside of the information processing apparatus 100 can be modified to store the retained image.

The function of the information processing apparatus 100 has been described above.
[2-4. Processing flow]
Next, the flow of display processing executed by the information processing apparatus 100 will be described with reference to FIGS. 11 and 12.

(Overall flow of display processing)
First, the overall flow of the display process will be described with reference to FIG. FIG. 11 is a flowchart showing the flow of display processing executed by the information processing apparatus according to the second embodiment.

(S101) The imaging unit 102 images the object M. A captured image of the object M captured by the imaging unit 102 is input to the control unit 103.
(S102) When the shutter operation is performed by the user, the control unit 103 causes the storage unit 105 to store the captured image input from the imaging unit 102 in response to the shutter operation as a retained image. On the other hand, when the shutter operation is not performed, the process of S102 is skipped. Note that the position of the information processing apparatus 100 at the time of capturing the captured image recorded in the storage unit 105 as the retained image is set as the reference position.

  (S103) The control unit 103 determines whether or not to display the retained image on the display unit 104. When the retained image is displayed on the display unit 104, the process proceeds to S104. On the other hand, when the retained image is not displayed on the display unit 104 (when the captured image is displayed on the display unit 104), the process proceeds to S109.

  (S104) The position information acquisition unit 101 calculates the movement direction and movement amount of the information processing apparatus 100 from the detected acceleration value. Information (movement information) indicating the movement direction and movement amount calculated by the position information acquisition unit 101 is input to the control unit 103. Further, information on the acceleration value detected by the position information acquisition unit 101 is also input to the control unit 103.

(S105, S106) The control unit 103 calculates the position of the information processing apparatus 100 based on the reference position from the input movement information. Further, the control unit 103 calculates the reference magnification A according to the above equation (1) based on the viewing angle θ (see FIG. 10B), the Z coordinate Z _P of the current position, and the width W of the retained image. Further, the control unit 103 sets the operation magnification B according to the enlargement operation or reduction operation by the user.

(S107) The control unit 103 calculates the display magnification C defined by the above equation (2). Next, the control unit 103 calculates the width C * W and the height C * H of the target area based on the display magnification C. Further, the control unit 103 calculates the center coordinates (X _P , Y _P ) of the target area based on the movement information (see the reference symbol V in FIG. 10C).

(S108) The control unit 103 reads the retained image from the storage unit 105. Next, the control unit 103 cuts out a partial image corresponding to the target area from the held image based on the width C * W and height C * H of the target area and the center coordinates (X _P , Y _P ) calculated in S107. Next, the control unit 103 causes the display unit 104 to display the cut out partial image. When the process of S108 is completed, the process proceeds to S110.

(S109) The control unit 103 causes the display unit 104 to display the captured image input from the imaging unit 102. When the process of S109 is completed, the process proceeds to S110.
(S110) If the operation for terminating the application program has not been performed, the process returns to S101. On the other hand, when an operation for terminating the application program is performed, the series of processes illustrated in FIG. 11 is terminated.

(Switching determination process flow)
Here, the flow of the switching determination process corresponding to the process of S103 illustrated in FIG. 11 will be further described with reference to FIG. FIG. 12 is a flowchart showing the flow of the switching determination process executed by the information processing apparatus according to the second embodiment. Note that the switching determination process illustrated in FIG. 12 is mainly executed by the switching determination unit 134 included in the control unit 103.

  Note that the switching determination unit 134 holds the acceleration value az in the Z-axis direction input from the position information acquisition unit 101 for each time Tz. Further, it is assumed that the switching determination unit 134 holds the acceleration value az from the current time to the time Ta. It is assumed that the times Tz and Ta are set in advance. Also, the acceleration value at the current time is expressed as az [n] (n = Ta / Tz), the acceleration value at the time before time Ta is expressed as az [1], and recorded from the time before time Ta to the current time. The acceleration values are expressed in order as az [j] (j = 1,..., N).

  (S111) The switching determination unit 134 calculates the acceleration value difference azz using the acceleration value az in the Z-axis direction among the acceleration values input from the position information acquisition unit 101. However, the acceleration value difference azz is defined by the following equation (3), for example.

  (S112) The switching determination unit 134 determines whether or not the acceleration value difference azz calculated in S111 is equal to or less than a threshold value azz_max. The threshold value azz_max is a value close to 0 set in advance. For example, the threshold value azz_max is experimentally set in consideration of an operation performed when a person looks at a paper surface using a magnifying glass or a loupe. When the acceleration value difference azz is equal to or smaller than the threshold value azz_max, the process proceeds to S113. On the other hand, when the acceleration value difference azz is not less than or equal to the threshold value azz_max, the process proceeds to S117.

  (S113) The switching determination unit 134 calculates the acceleration value change amount daz [j] (j = 1,..., N−1). However, the change amount daz [j] of the acceleration value is defined by the following equation (4) for each of j = 1,..., N−1.

(S114) The switching determination unit 134 extracts the maximum value dazz from the acceleration value change amount daz [j] (j = 1,..., N−1) calculated in S113.
(S115) The switching determination unit 134 determines whether or not the maximum value dazz extracted in S114 is equal to or less than a threshold value dazz_max. The threshold value dazz_max is a value close to 0 set in advance. For example, the threshold value dazz_max is experimentally set in consideration of an operation performed when a person looks at a paper surface using a magnifying glass or a loupe. When the acceleration value difference dazz is equal to or smaller than the threshold value dazz_max, the process proceeds to S116. On the other hand, if the acceleration value difference dazz is not equal to or less than the threshold value dazz_max, the process proceeds to S117.

  (S116) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a retained image. That is, the switching determination unit 134 sets the display method to retained image display. When the process of S116 is completed, the series of processes shown in FIG.

  (S117) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a captured image. That is, the switching determination unit 134 sets the display method to real time display. When the process of S117 is completed, the series of processes shown in FIG.

The flow of display processing executed by the information processing apparatus 100 has been described above.
[2-5. Modification # 1 (Switching Determination Based on Distance)]
Next, a modified example (modified example # 1) of the second embodiment will be described with reference to FIGS. Modification # 1 relates to a method of switching between real-time display and retained image display according to the distance between the information processing apparatus 100 and the object M. That is, in the modification # 1, the function of the switching determination unit 134 among the functions of the information processing apparatus 100 is modified.

(Switching judgment function)
The function of the switching determination unit 134 included in the information processing apparatus 100 according to Modification # 1 will be described with reference to FIGS. 13 and 14.

  FIG. 13 is a first diagram for describing a switching determination function included in the information processing apparatus according to the modification (modification # 1) of the second embodiment. FIG. 14 is a second diagram for explaining the switching determination function of the information processing apparatus according to the modification (modification # 1) of the second embodiment.

  Here, a method for calculating the distance between the information processing apparatus 100 and the object M from the change in the captured image will be considered. As an example, as illustrated in FIG. 13, a captured image I1 captured in a situation where the information processing apparatus 100 is in the position P1 and a captured image I2 captured in a situation where the information processing apparatus 100 is in the position P2 are used. A method for calculating the distance will be described. The position P1 is a position where the distance between the object M and the information processing apparatus 100 is (D + dd). The position P2 is a position where the distance between the object M and the information processing apparatus 100 is D.

  The switching determination unit 134 records the captured image I1. Furthermore, the switching determination unit 134 records the acceleration value az1 in the Z-axis direction when the captured image I1 is captured. Thereafter, the switching determination unit 134 stands by until the time Tb has elapsed. The time Tb is a preset value. For example, the time Tb is set to a time sufficiently shorter than the time for the person to move the information processing apparatus 100 (for example, 50 milliseconds).

  It is assumed that the position of the information processing apparatus 100 has moved to the position P2 when the time Tb has elapsed after the captured image I1 is captured. The switching determination unit 134 records the captured image I2 captured at this time. Furthermore, the switching determination unit 134 records the acceleration value az2 in the Z-axis direction when the captured image I2 is captured. Then, the switching determination unit 134 calculates the distance dd that the information processing apparatus 100 has moved from the position P1 to the position P2 based on the following formula (5).

  Next, the switching determination unit 134 extracts feature points p1i (i = 1, 2,..., N1) from the captured image I1. Then, the switching determination unit 134 calculates the distance between all the feature points, and sets the average value of the calculated distance as dcma.

  For example, as illustrated in FIG. 14, the switching determination unit 134 extracts feature points p11 and p12, and calculates a distance dca between the feature points p11 and p12. The switching determination unit 134 similarly calculates the distance dca for other feature points, and calculates an average value dcma of the calculated distance dca.

  Note that, as feature points extracted from the captured image I1, for example, end points, corners, branch points, and the like are used. As a method for extracting feature points, a method of binarizing the captured image I1, thinning it, and then detecting an end point or the like from the arrangement between adjacent pixels is applicable. Applicable examples include H.P.Moravec and C.Harris algorithms.

  When the coordinates of the feature point p1i (i = 1, 2,..., N1) are given by (xi, yi), the average distance dcma is given by the following equation (6). Similarly, the switching determination unit 134 extracts feature points p2k (k = 1, 2,..., N2) from the captured image I2. And the switching determination part 134 calculates the distance between all the feature points, and makes the average value of the calculated distance dcmb.

  The switching determination unit 134 that has calculated the average value dcma related to the captured image I1 and the average value dcmb related to the captured image I2 calculates the distance D using the average values dcma, dcmb, and the distance dd. The distance D is given by the following equation (7).

  The switching determination unit 134 that has calculated the distance D compares the distance D with a preset value D_max. When the distance D is less than or equal to the value D_max, the switching determination unit 134 sets the display method to retained image display. On the other hand, when the distance D is not less than the value D_max, the switching determination unit 134 sets the display method to real-time display.

  Thus, in Modification # 1, the display method is switched according to the distance D between the information processing apparatus 100 and the object M. Since the real-time display is performed in a situation where the distance D is greater than the value D_max, the processing load for cutting out the partial image corresponding to the imaging range from the retained image is reduced.

(Switching determination process flow)
Next, the flow of the switching determination process executed by the information processing apparatus 100 according to Modification # 1 will be described with reference to FIG. This switching determination process is a process corresponding to the process of S103 shown in FIG. FIG. 15 is a flowchart showing the flow of the switching determination process executed by the information processing apparatus according to the modification (modification # 1) of the second embodiment. The process illustrated in FIG. 15 is mainly executed by the switching determination unit 134 included in the control unit 103.

(S201) The switching determination unit 134 records the captured image I1. Furthermore, the switching determination unit 134 records the acceleration value az1 in the Z-axis direction when the captured image I1 is captured.
(S202) The switching determination unit 134 waits until the time Tb elapses. The time Tb is a preset value. For example, the time Tb is set to a time sufficiently shorter than the time for the person to move the information processing apparatus 100 (for example, 50 milliseconds).

(S203) The switching determination unit 134 records the captured image I2. Furthermore, the switching determination unit 134 records the acceleration value az2 in the Z-axis direction when the captured image I2 is captured.
(S204) The switching determination unit 134 calculates the distance dd that the information processing apparatus 100 has moved from the position P1 to the position P2 based on the above equation (5).

  (S205) The switching determination unit 134 extracts feature points p1i (i = 1, 2,..., N1) from the captured image I1. In addition, the switching determination unit 134 extracts feature points p2k (k = 1, 2,..., N2) from the captured image I2.

  (S206) The switching determination unit 134 calculates distances between all feature points of the captured image I1, and sets the average value of the calculated distances as dcma. When the coordinates of the feature point p1i (i = 1, 2,..., N1) are given by (xi, yi), the average distance dcma is given by the above equation (6). Similarly, the switching determination unit 134 calculates distances between all feature points for the captured image I2, and sets the average value of the calculated distances as dcmb.

  (S207) The switching determination unit 134 calculates the distance D using the average value dcma related to the captured image I1, the average value dcmb related to the captured image I2, and the distance dd. At this time, the switching determination unit 134 calculates the distance D based on the above equation (7).

  (S208) The switching determination unit 134 determines whether or not the distance D is equal to or less than the threshold value D_max. If the distance D is less than or equal to the threshold value D_max, the process proceeds to S209. On the other hand, if the distance D is not less than or equal to the threshold value D_max, the process proceeds to S210.

  (S209) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a retained image. That is, the switching determination unit 134 sets the display method to retained image display. When the process of S209 is completed, the series of processes illustrated in FIG.

  (S210) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a captured image. That is, the switching determination unit 134 sets the display method to real time display. When the process of S210 is completed, the series of processes illustrated in FIG.

The modification # 1 has been described above.
[2-6. Modification # 2 (switching determination based on focus state)]
Next, a modified example (modified example # 2) of the second embodiment will be described with reference to FIGS. 16 and 17. Modification # 2 relates to a method of switching between real-time display and retained image display according to the focus state of a captured image. That is, in the modification # 2, the function of the switching determination unit 134 among the functions of the information processing apparatus 100 is modified.

(Switching judgment function)
The function of the switching determination unit 134 included in the information processing apparatus 100 according to the modified example # 2 will be described with reference to FIG. FIG. 16 is a diagram for describing a switching determination function included in the information processing apparatus according to the modification (modification # 2) of the second embodiment.

  Here, a method for detecting the focus state of the captured image is considered. FIG. 16 schematically shows a captured image I1 in which the focus state is excellent, a captured image I3 in which the focus state is inferior, and a captured image I2 that is in the middle of the focus state.

  When in focus, the image is sharp, and the difference in pixel value between the pixels in the image and the surrounding pixels increases. On the other hand, when the focus is greatly deviated as in the captured image I3, the image is blurred, so that the difference in pixel value between the pixels in the image and the peripheral pixels is small. Therefore, when the difference between the pixel values between adjacent pixels is calculated for each captured image, and the average value is calculated and compared between the captured images I1, I2, and I3, the captured image I1 has the largest value, and the captured image I3 Is the smallest value. Modification # 2 proposes a method for detecting the focus state of a captured image using such properties.

  First, the switching determination unit 134 converts the captured image into a gray scale. For example, the switching determination unit 134 averages three pixel values acquired in the three RGB channels, and sets the average value as a pixel value g [x, y]. However, (x, y) is a pixel position coordinate. Note that the switching determination unit 134 may extract the maximum value from the three pixel values as the pixel value g [x, y]. Next, the switching determination unit 134 calculates a change amount dg [x, y] of the pixel value g [x, y]. The change amount dg [x, y] is given by the following equation (8), for example.

  The switching determination unit 134 that has calculated the change amount dg [x, y] for all the pixels included in the captured image calculates the average value Dg of the change amount dg [x, y] for all the pixels. Then, the switching determination unit 134 compares the average value Dg with a preset value Dg_max. The value Dg_max may be set to a value obtained from the retained image. When the average value Dg is equal to or less than the value Dg_max, the switching determination unit 134 sets the display method to retained image display. On the other hand, when the average value Dg is not less than or equal to the value Dg_max, the switching determination unit 134 sets the display method to real-time display.

  Thus, in the modified example # 2, the display method is switched from the real-time display to the retained image display at the timing when the focus is not achieved. For this reason, real-time display is performed while the subject is in focus, and the processing load for cutting out the partial image corresponding to the imaging range from the retained image is reduced. In addition, since a focused state is maintained regardless of the position of the information processing apparatus 100, a seamless operation feeling can be provided to the user.

(Switching determination process flow)
Next, the flow of the switching determination process executed by the information processing apparatus 100 according to the modified example # 2 will be described with reference to FIG. This switching determination process is a process corresponding to the process of S103 shown in FIG. FIG. 17 is a flowchart illustrating the flow of the switching determination process executed by the information processing apparatus according to the modification (modification # 2) of the second embodiment. The process illustrated in FIG. 17 is mainly executed by the switching determination unit 134 included in the control unit 103.

  (S231) The switching determination unit 134 converts the captured image into a gray scale. For example, the switching determination unit 134 averages three pixel values acquired in the three RGB channels, and sets the average value as a pixel value g [x, y].

  (S232) The switching determination unit 134 calculates a change amount dg [x, y] of the pixel value g [x, y]. For example, for the change amount dg [x, y], a difference in pixel value is calculated for each pixel from surrounding pixels, and the maximum value of the calculated difference is set as the change amount dg [x, y] (the above formula) (See (8)).

(S233) The switching determination unit 134 calculates the average value Dg of the change amount dg [x, y] for all the pixels included in the captured image.
(S234) The switching determination unit 134 determines whether or not the average value Dg is equal to or less than the threshold value Dg_max. If the average value Dg is equal to or less than the threshold value Dg_max, the process proceeds to S235. On the other hand, when the average value Dg is not less than or equal to the threshold value Dg_max, the process proceeds to S236.

  (S235) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a retained image. That is, the switching determination unit 134 sets the display method to retained image display. When the process of S235 is completed, the series of processes shown in FIG.

  (S236) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a captured image. That is, the switching determination unit 134 sets the display method to real time display. When the process of S236 is completed, the series of processes shown in FIG.

The modification # 2 has been described above.
[2-7. Modification # 3 (Switching Determination Based on Lightness State)]
Next, a modified example (modified example # 3) of the second embodiment will be described with reference to FIGS. 18 and 19. Modification # 3 relates to a method of switching between real-time display and retained image display according to the brightness state of the captured image. That is, in Modification # 3, the function of the switching determination unit 134 among the functions of the information processing apparatus 100 is modified.

(Switching judgment function)
The function of the switching determination unit 134 included in the information processing apparatus 100 according to the modification # 3 will be described with reference to FIG. FIG. 18 is a diagram for describing a switching determination function included in the information processing apparatus according to the modification (modification # 3) of the second embodiment.

  Here, a method for detecting the brightness state of a captured image is considered. FIG. 18 schematically shows a captured image I1 having an excellent brightness state, a captured image I3 having a poor brightness state, and a captured image I2 having a brightness state that is approximately between these images.

  When the information processing apparatus 100 approaches the object M excessively, the shadow of the information processing apparatus 100 overlaps the object M, and the brightness of the captured image decreases as shown in the captured image I3. Therefore, the switching determination unit 134 switches from real-time display to retained image display before the brightness decreases due to insufficient light quantity. The switching determination unit 134 obtains the brightness bp [x, y] for each pixel for all the pixels included in the captured image. However, (x, y) is a pixel position coordinate.

  For the pixel at (x, y), the maximum value of the three pixel values (R, G, B) acquired in the three RGB channels is expressed as Imax [x, y] and the minimum value is expressed as Imin [x, y]. To do. The switching determination unit 134 calculates the brightness bp [x, y] given by, for example, the following equation (9) or equation (10) for all pixels included in the captured image.

  The switching determination unit 134 obtains an average value Bp for the brightness bp [x, y] calculated for all the pixels included in the captured image. Then, the switching determination unit 134 compares the calculated average value Bp with a preset value Bp_max. The value Bp_max may be a value obtained from the retained image. When the average value Bp is equal to or less than the value Bp_max, the switching determination unit 134 sets the display method to retained image display. On the other hand, when the average value Bp is not less than or equal to the value Bp_max, the switching determination unit 134 sets the display method to real-time display.

  As described above, in the modified example # 3, the display method is switched from the real-time display to the retained image display at the timing when the light amount becomes insufficient. Therefore, real-time display is performed while the amount of light is sufficient, and the processing load for cutting out the partial image corresponding to the imaging range from the retained image is reduced. Moreover, since a favorable brightness state is maintained regardless of the position of the information processing apparatus 100, a seamless operation feeling can be provided to the user.

(Switching determination process flow)
Next, the flow of the switching determination process executed by the information processing apparatus 100 according to Modification # 3 will be described with reference to FIG. FIG. 19 is a flowchart illustrating the flow of the switching determination process executed by the information processing apparatus according to the modification (modification # 3) of the second embodiment. The process illustrated in FIG. 19 is mainly executed by the switching determination unit 134 included in the control unit 103.

  (S251) The switching determination unit 134 calculates the brightness bp [x, y] for each pixel of the captured image. For example, the switching determination unit 134 calculates the brightness bp [x, y] given by the above formula (9) or formula (10) for all the pixels included in the captured image.

(S252) The switching determination unit 134 calculates the average value Bp for the brightness bp [x, y] calculated for all the pixels included in the captured image.
(S253) The switching determination unit 134 determines whether or not the average value Bp is equal to or less than the threshold value Bp_max. If the average value Bp is equal to or less than the threshold value Bp_max, the process proceeds to S254. On the other hand, if the average value Bp is not less than or equal to the threshold value Bp_max, the process proceeds to S255.

  (S254) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a retained image. That is, the switching determination unit 134 sets the display method to retained image display. When the process of S254 is completed, the series of processes illustrated in FIG. 19 ends.

  (S255) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a captured image. That is, the switching determination unit 134 sets the display method to real time display. When the process of S255 is completed, the series of processes shown in FIG. 19 ends.

The modification # 3 has been described above.
[2-8. Modification # 4 (switching determination based on focus state and lightness state)]
Next, a modified example (modified example # 4) of the second embodiment will be described with reference to FIG. Modification # 4 relates to a method of combining the mechanism of modification # 2 and the mechanism of modification # 3. That is, in Modification # 4, the function of the switching determination unit 134 among the functions of the information processing apparatus 100 is modified. FIG. 20 is a flowchart showing the flow of the switching determination process executed by the information processing apparatus according to the modified example (modified example # 4) of the second embodiment. The process illustrated in FIG. 20 is mainly executed by the switching determination unit 134 included in the control unit 103.

  (S271) The switching determination unit 134 executes determination processing based on the focus state. At this time, the switching determination unit 134 executes the switching determination process according to the modification # 2 illustrated in FIG.

  (S272) The switching determination unit 134 determines whether or not to display a retained image. If the display method is set to hold image display in the process of S271, the process proceeds to S273. On the other hand, if the display method is set to real time display in the process of S271, the process proceeds to S276.

  (S273) The switching determination unit 134 executes determination processing based on the brightness state. At this time, the switching determination unit 134 executes the switching determination process according to the modification # 3 illustrated in FIG.

  (S274) The switching determination unit 134 determines whether or not to display a retained image. If the display method is set to hold image display in the process of S273, the process proceeds to S275. On the other hand, if the display method is set to real time display in the process of S273, the process proceeds to S276.

  (S275) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a retained image. That is, the switching determination unit 134 sets the display method to retained image display. When the process of S275 is completed, the series of processes shown in FIG.

  (S276) The switching determination unit 134 sets an image to be displayed on the display unit 104 as a captured image. That is, the switching determination unit 134 sets the display method to real time display. When the process of S276 is completed, the series of processes shown in FIG.

  As described above, in the modification # 4, the display method is switched from the real-time display to the retained image display at the timing when the focus is not achieved or the light quantity is insufficient. Therefore, real-time display is performed while the image is in focus and the amount of light is sufficient, and the processing load for cutting out the partial image corresponding to the imaging range from the retained image is reduced. Moreover, since a favorable focus state and brightness state are maintained regardless of the position of the information processing apparatus 100, a seamless operation feeling can be provided to the user.

The modification # 4 has been described above.
[2-9. Modification # 5 (Recording of captured image based on reduction determination)]
Next, a modified example (modified example # 5) of the second embodiment will be described with reference to FIGS. Modification # 5 relates to a method for selecting a retained image based on a comparison result of captured images captured in time series. That is, in Modification # 5, the function of the image recording unit 133 is modified among the functions of the information processing apparatus 100.

(Recorded image recording function)
The function of the image recording unit 133 included in the information processing apparatus 100 according to Modification # 5 will be described with reference to FIGS.

  FIG. 21 is a first diagram for describing a captured image recording function of an information processing apparatus according to a modification (modification # 5) of the second embodiment. FIG. 22 is a second diagram for describing the captured image recording function of the information processing apparatus according to the modification (modification # 5) of the second embodiment. FIG. 23 is a third diagram for describing the captured image recording function of the information processing apparatus according to the modification (modification # 5) of the second embodiment.

  So far, the description has been made on the assumption that the captured image is recorded in the storage unit 105 in accordance with the shutter operation of the user and is held as a retained image. In the modification # 5, a method is proposed in which the image recording unit 133 holds a certain amount of captured images, and selects captured images satisfying the set condition from them, and sets them as retained images.

  As shown in FIG. 21, the image recording unit 133 according to the modified example # 5 captures captured images I [1] and I [I [1] that have been captured before a preset time Te (for example, 2 seconds) from the current time. 2], ..., I [n] are held. Note that I [n] is a captured image captured at the current time, and I [1] is a captured image captured at a time before time Te.

  When the display method is switched from the real-time display to the retained image display, the image recording unit 133 extracts the captured image I [n−1] captured before the switching time (current time). Then, the image recording unit 133 searches a captured image that is not a reduced image of the extracted captured image I [n−1] retrospectively.

  In the example of FIG. 21, the captured image I [n−1],..., I [j + 1] is a reduced image of the captured image I [n−1]. Therefore, the image recording unit 133 detects the captured image I [j] as a captured image that is not a reduced image of the captured image I [n−1] (hereinafter, non-reduced image). In this case, the image recording unit 133 selects the captured image I [j + 1] captured after the captured image I [j], which is a non-reduced image, as a retained image. Then, the image recording unit 133 causes the storage unit 105 to store the selected captured image I [j + 1].

  Thus, by automating the recording of the captured image, a retained image can be obtained even when the user does not perform a shutter operation when the information processing apparatus 100 is brought close to the object M. As a result, it is possible to avoid the risk of forgetting the shutter operation or the risk of failing to perform the shutter operation and being unable to switch to holding image display. Further, since a retained image corresponding to a wide imaging range including the object M is obtained, a wide range in which the information processing apparatus 100 can move can be secured while maintaining the retained image display state.

(Determination of reduced image / non-reduced image)
Here, a method for determining whether or not the image is a reduced image will be described with reference to FIGS. 22 and 23.

  First, referring to FIG. In the example of FIG. 22, the captured image I [j + 1] and the captured image I [n−1] are compared. The image recording unit 133 extracts feature points from the captured image I [n−1]. As the feature points, for example, end points, corners, branch points, and the like are used. As a feature point extraction method, a method of binarizing the captured image I [n−1] and thinning it and then detecting an end point or the like from the arrangement between adjacent pixels can be applied. Applicable examples include H.P.Moravec and C.Harris algorithms.

  In the example of FIG. 22, five feature points p1,..., P5 are extracted. Information on the feature points p1,..., P5 is expressed by a vector extending from the origin O to the feature points p1,. Hereinafter, the feature points p1,..., P5 may be expressed as feature point vectors p1 (x1, y1),..., P5 (x5, y5), respectively. As shown in FIG. 22, the captured image I [n-1] is characterized by five feature point vectors p1 (x1, y1),..., P5 (x5, y5). Similarly, the image recording unit 133 extracts the feature points p6,..., P10 from the captured image I [j + 1], and uses the captured image I [[10, y10) with the feature point vectors p6 (x6, y6),. j + 1].

  The image recording unit 133 converts a feature point vector that is r (r <1) times the feature point vector p1 (x1, y1) extracted from the captured image I [n−1] to the feature point vector of the captured image I [j + 1]. Detection is performed from p6 (x6, y6),..., p10 (x10, y10). In the example of FIG. 22, the feature point vector p6 (x6, y6) is r1 (r1 = x6 / x1 = y6 / y1) times the feature point vector p1 (x1, y1). Therefore, the image recording unit 133 detects the feature point vector p6 (x6, y6) as a feature point vector that is r (r <1) times the feature point vector p1 (x1, y1).

  In the example of FIG. 22, the feature point vector p7 (x7, y7) is r2 (r2 = x7 / x2 = y7 / y2) times the feature point vector p2 (x2, y2). The feature point vector p8 (x8, y8) is r3 (r3 = x8 / x3 = y8 / y3) times the feature point vector p3 (x3, y3). The feature point vector p9 (x9, y9) is r4 (r4 = x9 / x3 = y9 / y3) times the feature point vector p4 (x4, y4). The feature point vector p10 (x10, y10) is r5 (r5 = x10 / x5 = y10 / y5) times the feature point vector p5 (x5, y5).

  Therefore, the image recording unit 133 detects the feature point vector p7 (x7, y7) as a feature point vector that is r (r <1) times the feature point vector p2 (x2, y2). Similarly, the image recording unit 133 detects a feature point vector p8 (x8, y8) as a feature point vector that is r (r <1) times the feature point vector p3 (x3, y3). Furthermore, the image recording unit 133 detects a feature point vector p9 (x9, y9) as a feature point vector that is r (r <1) times the feature point vector p4 (x4, y4). Then, the image recording unit 133 detects the feature point vector p10 (x10, y10) as a feature point vector that is r (r <1) times the feature point vector p5 (x5, y5).

  The image recording unit 133 uses the feature point vector that is r (r <1) times the feature point vector p1 (x1, y1) extracted from the captured image I [n−1] as the feature point of the captured image I [j + 1]. Calculate how many exist in a vector. Then, the image recording unit 133 determines whether or not the calculated number q of feature point vectors is equal to or greater than a set value Rv * nv. However, the value Rv (Rv <1; for example, Rv = 0.7) is a preset value. nv is the number of feature points extracted from the captured image I [n−1].

  Furthermore, the image recording unit 133 calculates an average value r0 of r1, r2,... And determines whether the difference between r1, r2,. When the number q is equal to or greater than the value Rv * nv and the difference between r1, r2,... And r0 is within ± σ, the image recording unit 133 indicates that the captured image I [j + 1] is captured image I [n −1] reduced image. In the example of FIG. 23, the feature point vectors p11 (x11, y11),..., P15 (x15, y15) of the captured image I [j] are the feature point vectors p1 (x1, y1) of the captured image I [n−1]. ),..., R (r <1) times p5 (x5, y5). Therefore, the image recording unit 133 determines that the captured image I [j] is not a reduced image of the captured image I [n−1] (determines that it is a non-reduced image).

The method for determining whether or not the image is a reduced image has been described above.
(Flow of captured image recording process)
Next, the flow of a captured image recording process executed by the information processing apparatus 100 according to the modified example # 5 will be described with reference to FIGS. 24 and 25.

(Overall flow of display processing)
First, the overall flow of the display process will be described with reference to FIG. FIG. 24 is a flowchart illustrating a flow of display processing executed by the information processing apparatus according to the modification (modification # 5) of the second embodiment.

(S301) The imaging unit 102 images the object M. A captured image of the object M captured by the imaging unit 102 is input to the control unit 103.
(S302) The control unit 103 records the captured image in a buffer. For example, the control unit 103 uses the captured images I [1], I [2],..., I [n] captured before the preset time Te (for example, 2 seconds) from the current time as a buffer. Record and hold. Note that I [n] is a captured image captured at the current time, and I [1] is a captured image captured at a time before time Te.

  (S303) The control unit 103 determines whether or not to display the retained image on the display unit 104. When the retained image is displayed on the display unit 104 (when the display method is set to retained image display), the process proceeds to S304. On the other hand, when the retained image is not displayed on the display unit 104 (when the display method is set to real-time display), the process proceeds to S310.

  (S304) The control unit 103 selects an image to be stored in the storage unit 105 as a retained image from the captured images I [1], I [2], ..., I [n] recorded in the buffer. Then, the control unit 103 stores the selected image in the storage unit 105 as a retained image.

  (S305) The position information acquisition unit 101 calculates the movement direction and movement amount of the information processing apparatus 100 from the detected acceleration value. Information (movement information) indicating the movement direction and movement amount calculated by the position information acquisition unit 101 is input to the control unit 103. Further, information on the acceleration value detected by the position information acquisition unit 101 is also input to the control unit 103.

  (S306, S307) The control unit 103 calculates the position of the information processing apparatus 100 based on the reference position from the input movement information. Further, the control unit 103 calculates the reference magnification A according to the above equation (1) based on the viewing angle θ (see FIG. 10B), the Z coordinate ZP of the current position, and the width W of the retained image. Further, the control unit 103 sets the operation magnification B according to the enlargement operation or reduction operation by the user.

  (S308) The control unit 103 calculates the display magnification C defined by the above equation (2). Next, the control unit 103 calculates the width C * W and the height C * H of the target area based on the display magnification C. Further, the control unit 103 calculates the center coordinates (XP, YP) of the target region based on the movement information (see the reference symbol V in FIG. 10C).

  (S309) The control unit 103 reads the retained image from the storage unit 105. Next, the control unit 103 cuts out a partial image corresponding to the target area from the held image based on the width C * W and height C * H of the target area and the center coordinates (XP, YP) calculated in S308. Next, the control unit 103 causes the display unit 104 to display the cut out partial image. When the process of S309 is completed, the process proceeds to S311.

(S310) The control unit 103 causes the display unit 104 to display the captured image input from the imaging unit 102. When the process of S310 is completed, the process proceeds to S311.
(S311) If the operation for terminating the application program is not performed, the process returns to S301. On the other hand, when an operation for terminating the application program is performed, the series of processes illustrated in FIG. 24 is terminated.

(Flow of captured image recording process)
Here, the flow of the captured image recording process (holding image selection / recording process) will be further described with reference to FIG. This process corresponds to the process of S304 shown in FIG. FIG. 25 is a flowchart illustrating the flow of a captured image recording process executed by the information processing apparatus according to the modification (modification # 5) of the second embodiment. Note that the recording process shown in FIG. 25 is mainly executed by the image recording unit 133 included in the control unit 103.

  (S321) The image recording unit 133 determines whether or not the display before the display method determination processing is a retained image display. When the display method before the determination process is the retained image display, the series of processes illustrated in FIG. 25 ends. On the other hand, if the display method before the determination process is real-time display, the process proceeds to S322.

  (S322) The image recording unit 133 extracts the captured image I [n−1] captured before the time (current time) when the display method is switched from the real-time display to the retained image display. Then, the image recording unit 133 extracts feature points of the captured image I [n−1].

(S323) The image recording unit 133 sets an initial value of the index j. At this time, j = n−1.
(S324) The image recording unit 133 updates the value of the index j. At this time, j = j-1.

  (S325) The image recording unit 133 determines whether the index j is greater than 1. If the index j is greater than 1 (j> 1), the process proceeds to S327. On the other hand, if the index j is not greater than 1, the process proceeds to S326.

(S326) The image recording unit 133 selects the captured image I [1] recorded in the buffer. When the process of S326 is completed, the series of processes illustrated in FIG.
(S327) The image recording unit 133 extracts feature points of the captured image I [j].

  (S328) The image recording unit 133 compares the feature point of the captured image I [n-1] with the feature point of the captured image I [j]. For example, the image recording unit 133 confirms whether or not the captured image I [j] is a reduced image of the captured image I [n−1] from the comparison of the feature points by the determination method illustrated in FIGS. To do.

  (S329) If the captured image I [j] is a reduced image of the captured image I [n-1], the process proceeds to S330. On the other hand, if the captured image I [j] is not a reduced image of the captured image I [n−1], the process returns to S324.

  (S330) The image recording unit 133 selects the captured image I [j + 1] recorded in the buffer. Then, the image recording unit 133 stores the selected captured image I [j + 1] in the storage unit 105 as a retained image. When the process of S330 is completed, the series of processes shown in FIG.

The modification # 5 has been described above.
[2-10. Modification # 6 (Recording of Captured Image Based on Acceleration Determination)]
Next, a modified example (modified example # 6) of the second embodiment will be described with reference to FIGS. Modification # 6 relates to a method for selecting a retained image in accordance with a change in acceleration value. That is, in Modification # 6, the function of the image recording unit 133 among the functions of the information processing apparatus 100 is modified.

(Recorded image recording function)
The function of the image recording unit 133 included in the information processing apparatus 100 according to Modification # 6 will be described with reference to FIGS.

  FIG. 26 is a first diagram for describing a captured image recording function of an information processing apparatus according to a modification (modification # 6) of the second embodiment. FIG. 27 is a second diagram for describing the captured image recording function of the information processing apparatus according to the modification (modification # 6) of the second embodiment.

  In the modified example # 6, similarly to the modified example # 5, the image recording unit 133 holds a certain amount of captured images, and selects a captured image satisfying the set condition from among the captured images and makes it a retained image. suggest. However, in Modification # 5, a method of selecting a captured image based on a change in the captured image is proposed, but in Modification # 6, a method of selecting a captured image based on a change in acceleration is proposed.

  As shown in FIG. 26, the image recording unit 133 according to the modified example # 6 captures captured images I [1] and I [1] taken from the current time to a time Te (for example, 2 seconds) before the preset time. 2], ..., I [n] are held. Note that I [n] is a captured image captured at the current time, and I [1] is a captured image captured at a time before time Te. Furthermore, the image recording unit 133 holds the acceleration values ax, ay, and az acquired from the position information acquisition unit 101 in association with the captured images I [1], I [2], ..., I [n].

  When the display method is switched from the real-time display to the retained image display, the image recording unit 133 captures the captured image I [k] (k = 1,..., N−1) captured before the switching time (current time). ) Are referred to. Further, the image recording unit 133 investigates the change amount dA of the acceleration values ax, ay, and az while going back to the past.

  At this time, the image recording unit 133 calculates the changes dax, day, and daz of the acceleration values ax, ay, and az according to the following equations (11) to (13). Note that ax [j] represents the acceleration value in the X-axis direction at the imaging time of the captured image I [j]. Similarly, ay [j] represents the acceleration value in the Y-axis direction at the imaging time of the captured image I [j]. az [j] represents the acceleration value in the Z-axis direction at the imaging time of the captured image I [j]. Then, the image recording unit 133 selects the maximum values of the change amounts dax, day, and daz, and sets the selected maximum value as the change amount dA.

  The image recording unit 133 causes the storage unit 105 to store the captured image captured after the time when the change amount dA exceeds the preset value dA_max as a retained image. In the example of FIG. 26, since the change amount dA of the acceleration value exceeds the value dA_max at the imaging time of the captured image I [j], the captured image I [j + 1] is recorded in the storage unit 105 as a retained image.

  Thus, by automating the recording of the captured image, a retained image can be obtained even when the user does not perform a shutter operation when the information processing apparatus 100 is brought close to the object M. As a result, it is possible to avoid the risk of forgetting the shutter operation or the risk of failing to perform the shutter operation and being unable to switch to holding image display. Further, since a retained image corresponding to a wide imaging range including the object M is obtained, a wide range in which the information processing apparatus 100 can move can be secured while maintaining the retained image display state.

(Flow of captured image recording process)
Next, the flow of a captured image recording process executed by the information processing apparatus 100 according to Modification # 6 will be described with reference to FIG. This process corresponds to the process of S304 shown in FIG. However, the acceleration value is recorded in the buffer together with the captured image. FIG. 28 is a flowchart illustrating the flow of a captured image recording process executed by the information processing apparatus according to the modification (modification # 6) of the second embodiment.

  (S351) The image recording unit 133 determines whether or not the display before the display method determination process is a retained image display. When the display method before the determination process is the retained image display, the series of processes illustrated in FIG. 28 ends. On the other hand, if the display method before the determination process is real-time display, the process proceeds to S352.

(S352) The image recording unit 133 sets an initial value of the index j. At this time, j = n−1.
(S353) The image recording unit 133 updates the value of the index j. At this time, j = j-1.

  (S354) The image recording unit 133 determines whether the index j is greater than 1. If the index j is greater than 1 (j> 1), the process proceeds to S356. On the other hand, if the index j is not greater than 1, the process proceeds to S355.

(S355) The image recording unit 133 selects the captured image I [1] recorded in the buffer. When the process of S355 is completed, the series of processes illustrated in FIG.
(S356, S357) The image recording unit 133 calculates the change amounts dax, day, daz of the acceleration values ax, ay, az according to the above formulas (11) to (13). Then, the image recording unit 133 selects the maximum value dA of the change amounts dax, day, and daz.

  (S358) The image recording unit 133 determines whether or not the maximum value dA is larger than the set value dA_max. If the maximum value dA is greater than the set value dA_max, the process proceeds to S359. On the other hand, when the maximum value dA is not larger than the set value dA_max, the process returns to S353.

  (S359) The image recording unit 133 selects the captured image I [j + 1] recorded in the buffer. Then, the image recording unit 133 stores the selected captured image I [j + 1] in the storage unit 105 as a retained image. When the process of S359 is completed, the series of processes illustrated in FIG.

The modification # 6 has been described above.
The second embodiment has been described above.
The preferred embodiments have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious for a person skilled in the art that various variations and modifications can be conceived within the scope of the claims, and such variations and modifications are naturally understood by the technical scope of the present invention. It goes without saying that it belongs to a range.

  For example, when the display is switched to the retained image display, a display method of displaying other information superimposed on the retained image is conceivable. As an example, consider the case where a map is the object M. The map includes information on various roads and facilities. For this reason, there is a case where desired information cannot be quickly found by simply enlarging and displaying the information processing apparatus 100 close to the map.

  Therefore, when switching to retained image display, if a method of applying image processing to the retained image and highlighting a specific color is applied, a specific road or facility on the map is highlighted, and the highlighted portion is displayed. The desired information can be quickly found by using as a landmark. For example, if the setting is to highlight green, the expressway and its entrance display are highlighted, and it is expected that the location identification and the region discrimination can be made efficient. Naturally, such a modified example also belongs to the above technical scope.

DESCRIPTION OF SYMBOLS 10 Information processing apparatus 11 Imaging part 12 Storage part 13 Display part 14 Control part 21 Target object 22 Imaging range 23a, 23b Image 24 Shadow D1, D2 Distance

Claims (13)

An imaging unit for imaging an object;
A storage unit for storing images;
An acceleration sensor for detecting acceleration;
An image of the object captured by the imaging unit is displayed on the screen in real time, at least one image is stored in the storage unit, and output from the acceleration sensor in a direction toward the object. When the condition that the change amount of the value is smaller than the set first value is satisfied, the image stored in the storage unit is read, and the partial image corresponding to the current imaging range is extracted from the read image And a control unit that switches the display of the screen to the partial image. An imaging unit for imaging an object;
A storage unit for storing images;
An image of the object imaged by the imaging unit is displayed on a screen in real time, at least one image is stored in the storage unit, and an index value indicating a degree of blur of the image captured by the imaging unit is When the condition that it is larger than the set third value is satisfied, the image stored in the storage unit is read, and a partial image corresponding to the current imaging range is extracted from the read image, and the screen And a control unit that switches the display to the partial image. Wherein, when said partial image no longer meets the pre Kijo matter in a state of being displayed on the screen, the screen display of the image of the object to the imaging unit at the present time are captured The information processing apparatus according to claim 1 or 2. An imaging unit for imaging an object;
A storage unit for storing images;
A second value in which an image of the object imaged by the imaging unit is displayed on a screen in real time, at least one image is stored in the storage unit, and a distance from the object is set The image stored in the storage unit is read out, a partial image corresponding to the current imaging range is extracted from the read image, and the display on the screen is converted into the partial image. Control part to switch
An information processing apparatus. An imaging unit for imaging an object;
A storage unit for storing images;
The image of the object imaged by the imaging unit is displayed on the screen in real time, at least one of the images is stored in the storage unit, and the brightness of the image captured by the imaging unit is set. When the condition that the value is smaller than the value is satisfied, the image stored in the storage unit is read out, and the partial image corresponding to the current imaging range is extracted from the read image, and the display of the screen is performed as the partial image. Control unit to switch to
An information processing apparatus. The control unit reads the image stored in the storage unit when the additional condition that the brightness of the image captured by the imaging unit is smaller than the set fourth value is satisfied, and reads the image stored in the read image from the read image The information processing apparatus according to claim 2, wherein a partial image corresponding to a current imaging range is extracted and the display on the screen is switched to the partial image . Wherein the control unit holds by an amount set an image of the object being imaged in chronological order by the imaging unit, before if satisfying Kijo matter, before from among the images held Kijo 2. An image which is a reduced image of an image held immediately before satisfying the condition is selected, a partial image corresponding to the current imaging range is extracted from the selected image, and the display on the screen is switched to the partial image. The information processing apparatus according to any one of to 5 . Wherein the control unit, the information processing apparatus according to claim 7 for selecting the oldest image in the image which is a reduced image of the image held immediately before meeting the pre Kijo matter from the images that the holding . Prior Symbol controller, holding the output value of the acceleration sensor at the time of imaging of the image holding by an amount set an image of the object being imaged in reverse chronological order, and held by the image pickup unit in association with the image and, if the previous met Kijo matter, from among the images held, select an image corresponding to said output value change amount is larger than the fifth value set, the selected image The information processing apparatus according to claim 1, wherein a partial image corresponding to a current imaging range is extracted and the display on the screen is switched to the partial image. Camera, a memory, processor, and the processor of the computer having an acceleration sensor for detecting acceleration,
An image of the object captured by the camera is displayed on the screen in real time, at least one of the images is stored in the memory, and a change in a value output from the acceleration sensor in a direction toward the object When the condition that the amount is smaller than the set value is satisfied, the image stored in the memory is read, and the partial image corresponding to the current imaging range is extracted from the read image, and the display of the screen is performed. A program for executing a process of switching to the partial image. A computer having a camera, a memory, and a processor;
An image of an object captured by the camera is displayed on the screen in real time, at least one of the images is stored in the memory, and an index value indicating a degree of blur of the image captured by the camera is set from a set value. The image stored in the memory is read out, the partial image corresponding to the current imaging range is extracted from the read image, and the display on the screen is switched to the partial image. A program that executes A computer having a camera, a memory, and a processor;
An image of an object imaged by the camera is displayed on a screen in real time, at least one image is stored in the memory, and a condition that a distance from the object is smaller than a set value. If the condition is satisfied, the image stored in the memory is read out, the partial image corresponding to the current imaging range is extracted from the read image, and the display on the screen is switched to the partial image.
A program that executes processing. A computer having a camera, a memory, and a processor;
An image of the object imaged by the camera is displayed on the screen in real time, at least one image is stored in the memory, and a condition that the brightness of the image imaged by the camera is smaller than a set value If the condition is satisfied, the image stored in the memory is read out, the partial image corresponding to the current imaging range is extracted from the read image, and the display on the screen is switched to the partial image.
A program that executes processing.

Images