JPWO2013099025A1 - Information processing apparatus, information processing method, information processing program, and information recording medium - Google Patents

Abstract

Provided is an information processing apparatus and the like that can easily image information described in an imaging object such as a document with high image quality, a wide range, and high resolution. In the portable smartphone S, at least a part of the imaging object is imaged, and the camera 9 that outputs frame data corresponding to the frame image corresponding to the captured at least a part, and the movement of the portable smartphone S The CPU 1 sequentially performs the alignment processing for aligning a plurality of frame images captured continuously in correspondence with the continuous imaging of the frame images using the respective frame images, and the aligned frame images. , And a CPU 1 that synthesizes a composite image corresponding to the imaging object and a composite image having a higher image quality or a wider range than each frame image captured by the camera 9.

Description

  The present invention belongs to a technical field of an information processing apparatus, an information processing method, an information processing program, and an information recording medium. More specifically, a portable information processing apparatus including an imaging unit such as a camera, an information processing method executed in the information processing apparatus, a program for the information processing apparatus, and an information recording medium on which the program is recorded It belongs to the technical field.

  In the information-oriented society in recent years, he wants to digitally image information written as letters, symbols or figures on paper (paper) etc. in high quality and high resolution even in the daily lives of individuals other than work. There are still many requests. Here, for imaging as described above, it is common to use a so-called scanner. As a scanner in this case, for example, a so-called flatbed scanner is generally used.

  However, it is often not convenient to have such a scanner for personal use other than business, from the viewpoint of cost, installation location, handling complexity, and the like. In recent years, the number of personal computers that do not have a personal computer has increased. Therefore, an environment in which a personal computer as a main body to which the scanner is connected does not exist in the first place has increased. Furthermore, imaging of information described in a book, magazine, or large paper that generally requires a large flatbed scanner is often more difficult due to the circumstances described above.

  On the other hand, so-called digital cameras have become common as imaging devices that can be easily handled by individuals. However, currently available digital cameras often have insufficient image quality (for example, the number of pixels, etc.) to replace them with scanners. Even if an attempt is made to use a digital camera instead of a scanner, if a general individual handles the camera, the camera may fail to capture the image due to, for example, camera shake or inappropriate lighting. Many.

  On the other hand, recently, everyone is carrying portable information devices with built-in cameras, such as so-called smartphones and tablet information terminals, and they are often worn all the time. . In addition, the imaging capability, image processing capability, and network connection capability of the camera included in the information device have been greatly improved in recent years, and rapid improvement is expected in the future.

  Apart from these, in particular in the field of medical examination image synthesis, etc., a technology has been developed that performs non-rigid (non-linear) alignment processing of multiple images obtained by imaging the same imaging object. (For example, see Non-Patent Document 1 below).

"Automated, Non-linear Registration Between 3-Dimensional Brain Map and Medical Head Image", pp175-183, "MEDICAL IMAGING TECHNOLOGY", Vol. 16, No . 3, May 1998

  As described above, there is currently an effective method that can be easily realized even by individuals, despite the fact that there are many requests to image information written in documents with high image quality and high resolution. Absent. Therefore, establishment of such an imaging method is desired.

  Therefore, the present invention has been made in view of the above-mentioned demands, and the problem is to image information described in an imaging object such as a document with high image quality, wide range, and high resolution, and more easily. It is to provide an information processing apparatus capable of performing the information processing, an information processing method executed in the information processing apparatus, a program for the information processing apparatus, and an information recording medium on which the program is recorded.

  In order to solve the above-described problem, the invention according to claim 1 is a frame corresponding to at least a part of the imaged object in an information processing apparatus such as a portable smartphone. An imaging unit such as a camera that outputs frame image information corresponding to an image, and a plurality of the frame images continuously captured by the imaging unit as the portable information processing device moves are aligned. Alignment means such as a CPU that sequentially performs alignment processing corresponding to continuous imaging of the frame image using a plurality of pieces of frame image information corresponding to the frame images, and the aligned frames Using the frame image information corresponding to each image, a composite image corresponding to the object to be imaged and captured by the imaging means Comprising a synthesizing unit such as a CPU performs the synthesis process for synthesizing a high-quality or wide range of the composite image than each of said frame images, a.

  In order to solve the above-described problem, the invention according to claim 19 is an information processing apparatus such as a portable smartphone, and images at least a part of the imaging target and corresponds to the captured at least part. In an image processing method executed in an information processing apparatus including an imaging unit such as a camera that outputs frame image information corresponding to a frame image to be performed, the imaging unit continuously moves with the movement of the portable information processing apparatus. Positions for sequentially performing alignment processing for aligning a plurality of captured frame images using a plurality of pieces of frame image information corresponding to the respective frame images, corresponding to continuous imaging of the frame images Using the alignment step and frame image information corresponding to each of the aligned frame images, A corresponding composite image; and a synthesis step for synthesizing process for synthesizing a high-quality or wide range of the composite image than each of said frame image captured by the imaging means.

  In order to solve the above-described problem, the invention according to claim 20 is an information processing apparatus such as a portable smartphone, and images at least a part of an imaging target and corresponds to at least a part of the captured image. A computer included in an information processing apparatus including an image capturing unit such as a camera that outputs frame image information corresponding to a frame image to be captured was continuously captured by the image capturing unit as the portable information processing apparatus moved. Alignment means for sequentially performing alignment processing for aligning a plurality of the frame images, using a plurality of pieces of frame image information corresponding to the respective frame images, corresponding to continuous imaging of the frame images; And corresponding to the imaging object by using frame image information corresponding to each of the aligned frame images. Synthesizing means for synthesizing process for synthesizing a high-quality or wide range of the composite image than each of said frame image captured by the imaging means an adult image, to function as a.

  In order to solve the above-described problem, the invention according to claim 21 is an information processing apparatus such as a portable smart phone, which captures at least a part of an imaging target and corresponds to the captured at least part. A computer included in an information processing apparatus including an image capturing unit such as a camera that outputs frame image information corresponding to a frame image to be captured was continuously captured by the image capturing unit as the portable information processing apparatus moved. Alignment means for sequentially performing alignment processing for aligning a plurality of the frame images, using a plurality of pieces of frame image information corresponding to the respective frame images, corresponding to continuous imaging of the frame images; And corresponding to the imaging object by using frame image information corresponding to each of the aligned frame images. An image processing program that functions as a synthesis unit that performs a synthesis process for synthesizing a synthesized image that has a higher image quality or a wider range than each of the frame images captured by the imaging unit is recorded in a readable manner by the computer. Has been.

  According to the invention according to any one of claims 1 or 19 to 21, alignment of a plurality of frame images continuously captured as the portable information processing apparatus moves is performed. In addition to sequentially performing corresponding to the continuous imaging, a high-quality image or a wide range of synthesized images are synthesized using a plurality of frame image information corresponding to each aligned frame image, so that a high-quality image corresponding to the imaging object Alternatively, a wide range of synthesized images can be synthesized using a portable information processing apparatus.

  In addition, since alignment processing and synthesis processing are performed using a plurality of frame image information obtained by continuous imaging accompanying the movement of the information processing apparatus, a higher image quality or A wide range of synthesized images can be synthesized.

  In order to solve the above-described problem, the invention according to claim 2 is the information processing apparatus according to claim 1, wherein the alignment unit corresponds to a whole frame image corresponding to the whole of the imaging object. Using the whole frame image information and a plurality of the frame image information corresponding to each of the frame images respectively corresponding to the part, the alignment processing is sequentially performed corresponding to the continuous imaging. The

  According to the invention described in claim 2, in addition to the operation of the invention described in claim 1, a plurality of pieces of frame image information respectively corresponding to a part of the imaging object and whole frame image information are used. Since the alignment process is sequentially performed corresponding to continuous imaging, the alignment process for each frame image can be performed more accurately.

  In order to solve the above-described problem, the invention according to claim 3 is the information processing apparatus according to claim 1 or 2, wherein the alignment means uses a plurality of the frame image information, The rigid body is configured to sequentially perform the alignment process.

  According to the invention described in claim 3, in addition to the operation of the invention described in claim 1 or 2, the non-rigid body alignment processing is sequentially performed using a plurality of frame image information output from the imaging unit. Therefore, it is possible to synthesize a high-quality image or a wide range of synthesized images with high versatility.

  In order to solve the above problem, an invention according to claim 4 corresponds to the frame image after the alignment processing in the information processing apparatus according to any one of claims 1 to 3. Based on imaged position range information indicating the position or range in the imaging object occupied by the part, position range information indicating the position or range of the part on the imaging object to be continuously imaged CPU for generating position range information generating means such as a CPU for generating and guidance information for guiding the movement of the information processing apparatus to a position corresponding to the position range information based on the generated position range information Guidance information generating means, and a notification means such as a display for notifying the generated guidance information to the outside of the information processing apparatus.

  According to the invention described in claim 4, in addition to the operation of the invention described in any one of claims 1 to 3, the position range information is generated based on the imaged position range information, and the generation is performed. In order to capture a frame image necessary for synthesizing a high-quality image or a wide range of synthesized images, information on the movement of the information processing device is generated based on the position range information thus generated and notified to the outside. The portable information processing apparatus can be moved accurately and quickly to a required position.

  In order to solve the above-mentioned problem, the invention according to claim 5 is the information processing apparatus according to any one of claims 1 to 3, wherein the imaging of the part to be continuously imaged is performed. Position range information generating means such as a CPU for generating position range information indicating a position or range on an object based on the captured frame image, and the position range information based on the generated position range information A guide information generating means such as a CPU for generating guide information for guiding the movement of the information processing device to a position corresponding to the information, a display for notifying the generated guide information to the outside of the information processing device, etc. And a notification means.

  According to the invention described in claim 5, in addition to the operation of the invention described in any one of claims 1 to 3, guidance information for movement guidance of the information processing device based on the position range information. Since the information is generated and notified to the outside, it is possible to accurately and quickly move the portable information processing apparatus to a position necessary for capturing a frame image necessary for synthesizing a high-quality image or a wide range of synthesized images. it can.

  In order to solve the above-described problem, the invention according to claim 6 is the information processing apparatus according to claim 4 or 5, wherein the synthesizing unit is configured to generate the frame image corresponding to the captured frame image. The composition processing based on information is sequentially performed corresponding to continuous imaging of the frame images, and the position range information generation means outputs composite image information corresponding to the composite image synthesized to an external image quality evaluation means. The acquired evaluation result information, comprising: an external output means such as a communication interface; and an acquisition means such as a communication interface for acquiring evaluation result information indicating an evaluation result by the image quality evaluation means for the output composite image information. Is further used to generate the position range information.

  According to the invention described in claim 6, in addition to the operation of the invention described in claim 4 or 5, the composition processing based on the frame image information corresponding to the captured frame image is performed. The position range information is generated based on the image quality of the synthesized image that has already been synthesized, and the position range information is generated by further using the evaluation result information for the synthesized image information by the external image quality evaluation means. Thus, it is possible to subsequently synthesize a higher-quality image or a wider range of synthesized images.

  In order to solve the above-described problem, the invention according to claim 7 is the information processing apparatus according to claim 5, wherein the position range information generation unit is configured to perform the positioning based on the frame image after the alignment process. It is configured to generate position range information.

  According to the seventh aspect of the invention, in addition to the operation of the fifth aspect of the invention, the position range information is generated based on the frame image after the alignment process, and thus the configuration completed within the information processing apparatus Thus, position range information can be generated, and a synthesized image having a higher image quality or a wider range can be synthesized thereafter.

  In order to solve the above problem, the invention according to claim 8 is the information processing apparatus according to claim 4 or 7, wherein the position range information generation means is used at least for the alignment processing. The CPU includes an accumulation unit such as a CPU that accumulates the number of the frame images for each pixel of the frame image after the alignment process, and the position range information generation unit is configured to add the position range information based on the accumulated number. Is configured to generate

  According to the invention described in claim 8, in addition to the operation of the invention described in claim 4 or 7, at least the number of frame images used for the alignment process is set to the number of frame images after the alignment process. Since the integration is performed for each pixel and the position range information is generated based on the integrated number, it is possible to generate the position range information capable of subsequently combining a higher-quality or wide range composite image. As the number of frame images in this case, for example, a number weighted according to the position in the frame image may be used.

  In order to solve the above problem, an invention according to claim 9 is the information processing apparatus according to any one of claims 4 to 8, wherein the guidance information is received by a carrier of the information processing apparatus. It is guidance information to be visually recognized, and further includes display means such as a display for displaying the guidance information.

  According to the invention described in claim 9, in addition to the operation of the invention described in any one of claims 4 to 8, the guide information is guide information visually recognized by a person who carries the information processing apparatus. Further, since the display means for displaying the guidance information is further provided, the user can easily move the information processing apparatus necessary for synthesizing a high-quality or wide-range synthesized image.

  In order to solve the above-described problem, the invention according to claim 10 is the information processing apparatus according to claim 9, wherein the guidance information includes, for each image quality, the image quality of the combined image after the combining process. It is comprised so that it may be the guidance information displayed by the aspect which can be identified.

  According to the invention described in claim 10, in addition to the operation of the invention described in claim 9, the guide information is guide information displayed in such a manner that the image quality in the composite image can be identified for each image quality. Therefore, the movement of the information processing apparatus necessary for synthesizing a high-quality image or a wide range of synthesized images can be easily performed according to the quality of the synthesized image.

  In order to solve the above problem, the invention according to claim 11 is the information processing apparatus according to claim 9 or 10, wherein the guide information corresponds to an image quality in the composite image after the composite processing. It is constituted so that it is the guidance information displayed by the color.

  According to the invention described in claim 11, in addition to the operation of the invention described in claim 9 or claim 10, the guide information is the guide information displayed in a color corresponding to the image quality in the composite image. The movement of the information processing apparatus necessary for synthesizing the image quality or a wide range of synthesized images can be easily performed according to the displayed color.

  In order to solve the above-mentioned problem, the invention according to claim 12 is the information processing apparatus according to any one of claims 9 to 11, wherein the display means displays the frame image. The guidance information is displayed so as to overlap the displayed frame image.

  According to the twelfth aspect of the invention, in addition to the action of the invention according to any one of the ninth to eleventh aspects, the guidance information is displayed so as to be superimposed on the frame image. The guidance information can be displayed in a manner that makes it easier to understand the content of the guidance information in relation to the imaging object that is being captured.

  In order to solve the above-described problem, an invention according to claim 13 is the information processing apparatus according to any one of claims 9 to 11, wherein the display means is provided on the entire imaging object. A corresponding whole frame image is displayed, and the guidance information is displayed over the whole frame image.

  According to the invention described in claim 13, in addition to the operation of the invention described in any one of claims 9-11, the guide information is displayed so as to be superimposed on the entire frame image. By displaying the guidance information superimposed on the frame image, it is possible to display the guidance information more easily.

  In order to solve the above-described problem, an invention according to claim 14 is the information processing apparatus according to any one of claims 9 to 13, wherein the guide information is the information processing that is carried around. It is comprised so that it may be the guidance information which shows the item of movement of an apparatus.

  According to the invention described in claim 14, in addition to the operation of the invention described in any one of claims 9-13, the guide information is guide information indicating specifications of movement of the information processing apparatus. Therefore, the information processing apparatus can be easily moved.

  In order to solve the above-described problem, the invention according to claim 15 is the information processing apparatus according to any one of claims 1 to 14, wherein at least one of the alignment process and the synthesis process. Interruption means such as a CPU for interrupting the processing in the information processing apparatus after one end, storage means such as a CPU for saving result information corresponding to the at least one of the completed processing results, and saved result information And a result information acquisition means such as a CPU for acquiring the information, and a restart means such as a CPU for restarting the interrupted processing using the acquired result information.

  According to the invention described in claim 15, in addition to the operation of the invention described in any one of claims 1 to 14, the information processing is performed after at least one of the alignment process and the synthesis process. Even when the process in the apparatus is interrupted, the process can be resumed using the stored result information, so that the degree of freedom in executing the alignment process and the synthesis process can be improved.

  In order to solve the above-described problem, the invention according to claim 16 is the information processing apparatus according to claim 15, wherein the result information includes a frame image corresponding to the at least one result until the interruption. Information is included, and the storage means includes external output means such as a communication interface for outputting the result information to an external image recognition means, and the result information acquisition means includes the frame recognized by the image recognition means. The result information including frame image information corresponding to the image is acquired.

  According to the invention of the sixteenth aspect, in addition to the action of the invention of the fifteenth aspect, the result information up to the interruption is recognized by the external image recognition means, the recognition result is acquired, and the process is resumed. Therefore, the synthesized image as the information processing apparatus can be synthesized with higher image quality or in a wider range.

  In order to solve the above-described problem, an invention according to claim 17 is the information processing apparatus according to any one of claims 1 to 16, wherein the imaging object is captured during imaging by the imaging unit. Illuminating means such as a light that illuminates a part or all of the information, and illumination control means such as a CPU that controls the illuminating means based on at least the result of the alignment process.

  According to the invention described in claim 17, in addition to the operation of the invention described in any one of claims 1 to 16, at least a part or all of the imaging object is obtained using the result of the alignment process. Therefore, it is possible to synthesize a higher-quality image or a wider range of synthesized images.

  In order to solve the above-described problem, an invention according to claim 18 is the information processing apparatus according to any one of claims 1 to 17, wherein the alignment unit is connected to the imaging object. The alignment processing is performed using the frame image information respectively output from the plurality of imaging units having different relative positions.

  According to the invention described in claim 18, in addition to the operation of the invention described in any one of claims 1 to 17, a plurality of imaging means having different relative positions with respect to the imaging object. Since the alignment processing is performed using the frame image information output from each of the images, it is possible to synthesize a higher-quality or wide-range composite image using a plurality of imaging units.

  As described above, according to the present invention, it is possible to synthesize a high-quality or wide-range synthesized image corresponding to an imaging object using a portable information processing apparatus. Further, by using a plurality of pieces of frame image information with different imaging conditions, it is possible to synthesize a higher-quality or wide range synthesized image.

  Therefore, it becomes possible to image the imaging object more easily with high image quality, wide range, and high resolution.

It is a block diagram which shows the schematic structure of the smart phone which concerns on 1st Embodiment. It is a flowchart which shows the image processing which concerns on 1st Embodiment. It is a figure which shows the example of a screen displayed in the image processing which concerns on 1st Embodiment, (a) is a figure which illustrates the document as an imaging target object, (b) is a display example at the time of an alignment process FIG. It is figure (i) which illustrates the position alignment process which concerns on 1st Embodiment, (a) is a figure which illustrates the state before position alignment, (b) is a figure which illustrates the position process in progress (i). (C) is a diagram (ii) illustrating the position alignment process. It is figure (ii) which illustrates the position alignment process which concerns on 1st Embodiment, (a) is a figure (iii) which illustrates during position alignment process, (b) is a figure which illustrates during position alignment process ( iv), (c) is a diagram (v) illustrating the alignment process, and (d) is a diagram (vi) illustrating the alignment process. It is figure (iii) which illustrates the alignment process which concerns on 1st Embodiment, (a) is a figure (i) which shows the other example of the alignment process, (b) is another example of the alignment process (C) is a diagram (iii) showing another example of the alignment process, and (d) is a diagram (iv) showing another example of the alignment process. It is a flowchart which shows the image processing which concerns on 2nd Embodiment.

Next, modes for carrying out the present invention will be described based on the drawings. In addition, each embodiment described below is an embodiment when the present invention is applied to a portable smartphone including a digital camera (hereinafter simply referred to as a camera) capable of continuous shooting and moving image capturing. is there. In addition, you may apply this invention with respect to the digital camera independent as an apparatus instead of the digital camera with which the said smart phone is equipped. In addition, a smartphone user according to the present invention is hereinafter simply referred to as a “carrier”.
(I) First Embodiment First, a first embodiment according to the present invention will be described with reference to FIGS. 1 is a block diagram showing a schematic configuration of the smartphone according to the first embodiment, and FIG. 2 is a flowchart showing image processing according to the first embodiment. FIG. 3 is a diagram illustrating an example of a screen displayed in the image processing according to the first embodiment, and FIGS. 4 to 6 are diagrams illustrating the alignment processing according to the first embodiment.

  The smartphone according to the first embodiment performs continuous shooting or moving image capturing of information described by using characters, symbols, figures, or the like on a document or a three-dimensional object as an example of an imaging object using the camera. . After that, the smartphone uses the image data obtained by the imaging for a plurality of frames by image processing to be described later according to the first embodiment, and performs image data mutual alignment processing and each image data after the alignment processing. A synthesis process to synthesize is executed. Thereby, the smartphone digitizes the information on the imaging object with high image quality or in a wide range.

  That is, as illustrated in FIG. 1, the smartphone S as an example of the “information processing apparatus” according to the first embodiment includes a CPU 1, a ROM (Read Only Memory) 2, a RAM (Random Access Memory) 3, and operation buttons. And an operation unit 4 including a touch panel, a display 5 including a liquid crystal display on the surface of the touch panel, a call control unit 6 to which a speaker 7 and a microphone 8 are connected, and an “imaging unit”. As an example, the camera 9 and a communication interface 10 including an antenna ANT for connecting to an external network (a network such as a wireless local area network (LAN), a dedicated line, the Internet, or a so-called 3G line) not shown in FIG. , Is configured. In the RAM 3, as various buffers necessary for executing image processing according to the first embodiment centered on the CPU 1, an alignment data buffer 30, a data amount buffer 31, a composite image buffer 32, and a wide area The image buffer 33, the current image buffer 34, and the aligned current image buffer 35 are formed as a volatile storage area.

  In the above configuration, the CPU 1 is an example of the “positioning unit”, the example of the “combining unit”, the example of the “position range information generating unit”, the example of the “guidance information generating unit”, the “integration” according to the first embodiment. It corresponds to an example of “means”, an example of “interrupt means”, an example of “save means”, an example of “result information acquisition means”, an example of “resume means”, and an example of “illumination control means”. The display 5 corresponds to an example of “notification means” and an example of “display means”, respectively. Further, the communication interface 10 corresponds to an example of “external output unit” and an example of “acquisition unit”, respectively.

  In this configuration, the communication interface 10 controls the exchange of data with the network via the antenna ANT under the control of the CPU 1. At this time, the communication interface 10 may be configured to control not only wireless data transmission / reception via the antenna ANT but also wired data transmission / reception via a wired LAN or a so-called USB (Universal Serial Bus), for example. it can.

  The call control unit 6 controls a voice call as the smartphone S using the microphone 8 and the speaker 7 under the control of the CPU 1. Furthermore, the operation part 4 produces | generates the operation signal corresponding to the said operation based on operation by the user of the smart phone S, and outputs it to CPU1. Thereby, CPU1 controls the smart phone S whole based on the said operation signal.

  On the other hand, in the ROM 2, a program or the like for processing as the smartphone S including image processing according to the first embodiment described later is recorded in a nonvolatile manner in advance. And CPU1 controls the process as the said smart phone S by reading the said program etc. from ROM2, and running it. In addition, the ROM 2 also stores data necessary for processing as the smartphone S, such as telephone number data and address data, in a nonvolatile manner. The RAM 3 temporarily stores data necessary for each of the buffers, and further temporarily stores other data necessary for processing as the smartphone S. Further, the display 5 displays information necessary for processing as the smartphone S to the user under the control of the CPU 1 in addition to information necessary for image processing according to the first embodiment.

  On the other hand, under the control of the CPU 1, the camera 9 continuously captures or captures the information on the document or the like, and acquires image data (digitized image data) corresponding to the captured image for each continuous capture. Or in the case of moving image imaging, it outputs to CPU1 continuously. Thus, the CPU 1 executes image processing according to the first embodiment using data stored in each buffer in the RAM 3.

  Next, each buffer in the RAM 3 will be specifically described.

  First, the alignment data buffer 30 in the RAM 3 stores coordinate data indicating a deformation state as a result of alignment processing described later. Next, the data amount buffer 31 in the RAM 3 is an image that is a target of the alignment process and the synthesis process in the image process according to the first embodiment for each pixel in the image captured by the camera 9 (described later). The number (nameless number) of the current image) is stored for each pixel in the composite image. Here, the larger the value stored in the data amount buffer 31, the higher the image quality of the pixel is improved by the alignment process and the synthesis process according to the first embodiment. For example, a value weighted according to the position in the image may be stored in the data amount buffer 31 for each pixel.

  On the other hand, the composite image buffer 32 in the RAM 3 adjusts the image data corresponding to the composite image formed with high image quality / wide range by the composite processing of the image processing according to the first embodiment in accordance with the progress of the composite processing. And memorize sequentially. On the other hand, the wide area image buffer 33 in the RAM 3 is used for the alignment process described later, and image data (frame) corresponding to the entire image such as a document imaged in the image processing according to the first embodiment. Image data). Here, the “whole image” in this case means, for example, one image obtained by capturing the entire document when the information described in one document is digitized.

  In addition, the current image buffer 34 in the RAM 3 stores image data for one frame that is the target of the alignment processing in the image processing according to the first embodiment at that time. In this case, an image corresponding to one frame of image data (frame image data) is referred to as a “current image”. Finally, the aligned current image buffer 35 in the RAM 3 stores image data for one frame that is the target of the composition process after the alignment process in the image processing according to the first embodiment.

  Next, the image processing according to the first embodiment will be specifically described with reference to FIGS.

  The image processing according to the first embodiment is started, for example, when a predetermined operation by the carrier is executed on the operation unit 4.

  When the image processing is started, as shown in FIG. 2, the CPU 1 first initializes the alignment data buffer 30 with data having a content meaning “no deformation” (step S1). Thereby, the alignment process which concerns on 1st Embodiment is started. Next, the CPU 1 initializes the data amount buffer 31 and the composite image buffer 32 to “zero”, respectively (step S2).

  Next, the CPU 1 notifies the wearer of the imaging of the whole image through a process such as executing a display on the display 5 to capture the whole image. As a result, when the entire image is captured by a user's operation, the CPU 1 inputs image data for one frame corresponding to the entire image to the wide area image buffer 33 (step S3).

  Next, the CPU 1 notifies the user of continuous shooting or moving image shooting by executing a display on the display 5 to start continuous shooting or moving image shooting from an arbitrary position of the imaging target. As a result, when the continuous shooting or the moving image shooting is executed by a user's operation, the CPU 1 inputs image data for one frame corresponding to the current image input along with the shooting to the current image buffer 34. (Step S4). As a result, the CPU 1 performs non-rigid registration processing according to the first embodiment on the image data stored in the wide area image buffer 33 (that is, image data corresponding to the entire image) and the current image buffer 34 at that timing. This is performed using the stored image data (step S5). At this time, the non-rigid body alignment processing compares the portion of the imaging target imaged as the current image with the portion of the entire image, and performs alignment of the current image so that they match as an image. It is processing. This alignment process is performed using the coordinate data stored in the alignment data buffer 30. For example, the alignment process is described in Non-Patent Document 1 described above. More specifically, the alignment process will be described in detail later with reference to FIGS.

  When the alignment processing of the non-rigid body with respect to the image data for one frame corresponding to the current image is completed (step S5), the CPU 1 stores the image data of the current image after the alignment processing in the aligned current image buffer 35. (Step S6). Further, the CPU 1 newly stores coordinate data indicating the deformation state as a result of the alignment process in step S5 in the alignment data buffer 30 (step S7).

  Next, the CPU 1 determines whether or not the image quality of the current image after the alignment processing has sufficient resolution as the high image quality desired as the image processing of the first embodiment (step S8). In the process of step S8, for example, the CPU 1 uses the data indicating the preset target resolution corresponding to the high image quality and the size data input in advance indicating the size of the entire imaging target, for example. Determine the resolution of the image. In this case, in order to digitize the entire object to be imaged with high image quality or in a wide range, it is necessary to capture more current images as the target resolution or size data is larger, and the higher the target resolution is, the more demanded each current image is. The resolution will be high. Here, “the resolution of the current image is high” usually means that the object is closer to the object to be imaged, and the resolution is high, but the imaging range is narrow. If it is determined in step S8 that the image quality of the current image after the alignment process does not have sufficient resolution (step S8; NO), the CPU 1 proceeds to the process of step S12 described later. On the other hand, if it is determined in step S8 that the image quality of the current image after the alignment process has a sufficient resolution (step S8; YES), the CPU 1 next stores the image data in the aligned current image buffer 35. It adds to the image data in the composite image buffer 32 (step S9). By the processing in step S9, the high-quality current image is added to the synthesized image that has been synthesized so far. As a result, the regional ratio of the composite image in the entire imaging target is enlarged.

  Next, in accordance with the alignment process in step S5 and the composition process in step S9, the CPU 1 sets the value (in the data amount buffer 31 corresponding to each pixel of the composite image corresponding to any pixel included in the current image ( The value of the number of current images subjected to registration processing and composition processing is incremented by “1” corresponding to the registration processing and composition processing (step S10).

  Next, the CPU 1 determines whether or not the value of the data amount buffer 31 corresponding to each pixel is a sufficient value for all the areas of the imaging target in the composite image after the process of step S9 (step S11). . In the process of step S11, for example, in the composite image obtained as a result of the image processing according to the first embodiment, the CPU 1 determines that a sufficient number of current images as a high image quality desired as the image processing have been obtained. It is determined based on the value for each pixel stored in the data amount buffer 31 whether or not it has been subjected to the alignment process and the synthesis process according to the first embodiment over the region. If the determination in step S11 has a sufficient value (number of current images) (step S11; YES), the imaging object has been digitized with high image quality. Therefore, the CPU 1 divides the pixel value of each pixel of the composite image stored in the composite image buffer 32 by the value corresponding to that pixel stored in the data amount buffer 31 at that time (step S14), and the result. Is stored in a memory (not shown) or the like, and the image processing according to the first embodiment is terminated. In the process of step S14, the pixel value of each pixel in the high-quality synthesized image finally synthesized as the image processing according to the first embodiment is divided by the number of current images necessary for the image processing. It is a process, for example, a process for ensuring consistency with the current image picked up after shooting.

  On the other hand, in the determination in step S11, a sufficient number of current images in the composite image after the process in step S9 are the targets of the alignment process and the composite process according to the first embodiment over the entire area of the imaging target. If there is not (step S11; NO), the CPU 1 obtains a value corresponding to each pixel constituting the current current image from the data amount buffer 31 to capture an additional current image (step S12). In this case, the CPU 1 acquires the coordinate data stored in the alignment data buffer 30 in order to identify the pixel that has already been subjected to the alignment process and the synthesis process according to the first embodiment in the current image, Using this, a process (inverse transform process) opposite to the non-rigid positioning process according to step S5 is performed. After that, the CPU 1 causes the color corresponding to the value corresponding to each pixel acquired by the process of step S12 to be superimposed on the current image and displayed on the display 5 as guidance information according to the first embodiment (step S13). As a result, the CPU 1 notifies the wearer of the area within the imaging target that requires further current image capturing. Thereafter, the CPU 1 returns to step S4, and repeats the processes of steps S4 to S13 for the newly captured current image. As a result, the CPU 1 captures a new current image and performs alignment processing using the image data corresponding to the captured current image until the determination result in step S11 becomes “YES” (see step S5 above). The subsequent synthesis process (see steps S8 to S11 above) is repeated.

  Next, an example of an image displayed on the display 5 as the process of step S13 in the process of the above-described repetition process will be described with reference to FIG.

  In FIG. 3, it is assumed that the imaging target is a document P on which characters such as those illustrated in FIG. FIG. 3B is a front view of the smartphone S including the display 5 and the operation buttons 4A to 4C belonging to the operation unit 4. In the process in which the image processing according to the first embodiment is performed using the document P as an imaging target, for example, as illustrated in FIG. 3B, the desired high image quality (resolution) as the image processing according to the first embodiment. ) Is displayed so as to be hatched in both directions (see step S13 in FIG. 2), for example (see step S8 in FIG. 2; see YES). On the other hand, for an area that has not reached the desired high image quality (see step S8 in FIG. 2; NO), the image quality (resolution that has been reached in the previous image processing (in other words, stored in the data amount buffer 31). Are displayed in a stepwise different state (see step S13 in FIG. 2). For example, in the case illustrated in FIG. 3B, the area of the original document P is displayed as it is for the area where the current image has not yet been captured. On the other hand, the first area that has not reached the resolution corresponding to the high image quality is displayed so as to be hatched in the lower right direction, for example. Furthermore, although the resolution is lower than that of the first area, one or more current images are captured and the areas to be subjected to the alignment process and the synthesis process according to the first embodiment (the image according to the first embodiment). The area where processing has been started is displayed so as to be hatched in the lower left direction, for example. In FIG. 3B, the area hatched in both directions is, for example, red, the area hatched in the lower right direction is orange, and the area hatched in the lower left direction is yellow, You may comprise so that the resolution which has reached | attained with a color may be distinguished and displayed. Then, the user who viewed the display as illustrated in FIG. 3B adds and captures a current image of the area that is hatched in the lower right direction or the area that is hatched in the lower left direction. Accordingly, the smartphone S is moved to capture a necessary current image (see step S4 in FIG. 2 after the second round).

  Next, the non-rigid positioning process according to step S5 will be specifically described with reference to FIGS.

  As described above, the alignment processing according to Step 5 compares the part of the imaging target imaged as the current image with the part in the entire image, and compares the current image so that the two match as the image. This is processing for performing alignment (deformation). Here, in the examples shown in FIGS. 4 to 6, the whole image GA shown on the left in FIG. 4A is used as a reference image, and the non-rigid body alignment processing is performed on the current image GT shown on the right in FIG. And In the examples shown in FIGS. 4 to 6, the imaging areas of the current image GT and the entire image GA are the same.

  As the alignment processing according to step S5, first, the CPU 1 divides the current image GT into a predetermined number as illustrated in FIG. In the case illustrated on the right side of FIG. 4B, the current image GT is divided into four divided images GTa to GTd, but in order to obtain higher image quality, the larger the number of divisions, the better. Next, the CPU 1 pays attention to one divided image as illustrated in FIG. In the case illustrated in FIG. 4C, the CPU 1 focuses on the divided image GTa.

  Next, as illustrated in FIG. 5A, the CPU 1 superimposes the divided image GTa of interest on the entire image GA. Note that the coordinate axes in FIG. 5 have the origin (0, 0) as the upper left corner of the area corresponding to each divided image in the entire image GA and the upper left corner of each divided image, and the right direction in FIG. The downward direction in FIG. 5 is the positive direction of the y coordinate axis. In FIG. 5A, when the divided image GTa is first superimposed on the entire image GA, the offset is (0, 0).

  Thereafter, as illustrated in FIG. 5B, the CPU 1 searches the entire image GA for a position (offset) where the divided image GTa and the content best match. In addition, in order to quantify the coincidence between the contents of images, for example, a method using mutual information (Mutual Information) or a target region (in the case of FIG. 5B, the region of the divided image GTa). A method using a sum of luminance differences (SAD (Sum of Absolute Difference)) is suitable. In the example of FIG. 5B, the CPU 1 obtains coordinate data (−2, +3) as the offset.

  Thereafter, the CPU 1 similarly searches the entire image GA for the position (offset) where the contents of the divided images GTb to GTd other than the divided image GTa illustrated in FIG. In the case illustrated in FIG. 5C, coordinate data (−2, +3) is used as the offset in the case of the divided image GTa, and coordinate data (+2, +3) is used as the offset in the case of the divided image GTb. In this case, the coordinate data (+4, −1) is obtained by the CPU 1 as the offset, and in the case of the divided image GTd, the coordinate data (−3, −1) is obtained by the CPU 1 as the offset.

  As a result of the series of processes described above, the amount by which the center point of each of the divided images GTa to GTd is to be moved as the alignment processing according to step S5 is exemplified as the offset on the right in FIG. Obtained respectively. This center point may be generally called “anchor”. The coordinate data collected by adding the amounts to which all anchors should be moved for one current image GT is the coordinate data sequentially stored in the alignment data buffer 30.

  In addition, the position of the entire current image GT is deformed as illustrated in FIG. 6A by so-called interpolation or extrapolation based on the amount of movement of the anchor of each of the divided images GTa to GTd. It is also possible to perform a matching process. In this case, the contents of the current image GT and the contents of the entire image GA can be matched more. Furthermore, movement or deformation (including rotation, enlargement / reduction, or trapezoid deformation) of the entire current image GT or the entire divided image GTa to the entire divided image GTd can be used.

  Further, by comparing the shape or position of the entire current image GT after deformation with the shape or position of the original whole image GA as a whole, a smartphone as illustrated in FIGS. 6B to 6D is illustrated. It is also possible to notify the wearer of the imaging range of the next current image GT by estimating the movement of the entire S. For example, in the case illustrated in FIG. 6B, it is estimated that the entire smartphone S has moved in the lower left direction, and in the case illustrated in FIG. 6C, the entire smartphone S has been rotated upward (waving his head up). I guess). In the case illustrated in FIG. 6D, it is estimated that the entire smartphone S has moved backward (away from the imaging target).

  As described above, according to the image processing according to the first embodiment, the alignment of a plurality of images that are continuously captured along with the movement of the mobile smartphone S is sequentially performed corresponding to the continuous imaging. In addition, a high-quality or wide-range composite image is synthesized using a plurality of image data corresponding to the aligned images (see FIG. 2), so a high-quality or wide-range composite image corresponding to the object to be imaged Can be synthesized using a portable smartphone S. In addition, since alignment processing and synthesis processing are performed using a plurality of image data obtained by continuous imaging accompanying the movement of the smartphone S, a higher image quality or a wider range of synthesis can be achieved by using a plurality of image data with different imaging conditions. Images can be synthesized (see FIG. 2). In order to obtain this effect, it is not always necessary to execute the processing of steps S11 to S14 in FIG.

  In other words, this remarkable effect can be achieved by combining high-quality images or a wide range of images without using a device other than the smartphone S carried by the wearer. That is, since data such as images is not recorded other than the smartphone S, anytime, anywhere, any person, the privacy for individuals, the security of related information for work, while ensuring the highest level, respectively It is possible to easily synthesize high-quality images or a wide range of images without taking them. This is an epoch-making value that was not available in previous devices.

  Furthermore, since the alignment processing is sequentially performed corresponding to continuous imaging using a plurality of current images corresponding to a part of the imaging object and the entire image (see FIG. 2), the alignment for each image is performed. Processing can be performed more accurately.

  Furthermore, since the non-rigid body alignment processing is sequentially performed using a plurality of image data output from the camera 9 (see step S5 in FIG. 2), a highly versatile and high-quality or wide-range composite image is synthesized. Can do.

  Further, since guidance information for moving guidance of the smartphone S is generated and notified to the outside based on the value of the data amount buffer 31 corresponding to each pixel constituting the current image (see step S12 and step S13 in FIG. 2). The smartphone S can be accurately and quickly moved to a position necessary for synthesizing a high-quality image or a wide range of synthesized images.

  Furthermore, the number of each image used for synthesizing the synthesized image is accumulated using the data amount buffer 31 for each pixel of the synthesized image, and the guide information corresponding to each pixel is obtained based on the accumulated number. Since it is generated (see step S12 in FIG. 2), it becomes possible to synthesize a higher-quality or wider-range synthesized image thereafter.

  Furthermore, since the guidance information notified by the process of step S13 in FIG. 2 is guidance information visually recognized by the user of the smartphone S, the movement of the smartphone S necessary for synthesizing a high-quality or wide-range composite image is performed. That person can easily do.

  In addition, in the case where the guidance information notified by the processing in step S13 in FIG. 2 is displayed for each color according to the resolution (see FIG. 3B), a smartphone necessary for synthesizing a high-quality or wide-range composite image. The movement of S can be easily performed according to the displayed color.

  Further, when the guidance information is displayed over the original whole image GA as illustrated in FIG. 3B, the guidance information is understood in more detail with respect to the captured imaging object. It can be displayed easily. In this case, when the smartphone S (camera 9) is moved, the part of the imaging target displayed on the display 5 is changed in real time. Thereby, the content of guidance information can be displayed more easily.

Furthermore, when displaying guidance information overlaid on the overall image GA, the guidance information is displayed overlaid on the overall image GA, so that the content of the guidance information can be displayed more easily. can do. In this case, even if the smartphone S (camera 9) is moved, the entire imaging object is always displayed on the display 5 (the display range of the entire imaging object is not changed). Then, as the smartphone S (camera 9) is moved to acquire an image, the colored portion (pattern) displayed over the imaging object in order to show the image quality on the display 5 changes in real time. It will be followed. Even in this case, the contents of the guidance information can be displayed more easily.
(II) Second Embodiment Next, a second embodiment which is another embodiment according to the present invention will be described with reference to FIG. FIG. 7 is a flowchart showing image processing according to the second embodiment. The same processing as the image processing according to the first embodiment shown in FIG. Omitted. The hardware configuration of the smartphone according to the second embodiment is basically the same as that of the smartphone S according to the first embodiment. Therefore, in the following description, the smartphone according to the second embodiment will be described using the same member numbers as those of the smartphone S according to the first embodiment.

  In the image processing according to the first embodiment described above, the alignment processing and synthesis processing for the current image GT and the generation processing of guidance information to be notified to the user are performed as a series of flows. On the other hand, in the image processing according to the second embodiment described below, after performing the alignment processing, guidance information generation processing for notifying the wearer is performed before performing the synthesis processing.

  That is, as shown in FIG. 7A, in the image processing according to the second embodiment, the CPU 1 performs the same step S1 as the image processing according to the first embodiment as the alignment processing and composition processing according to the image processing. After performing Step S8, Step S10 to Step S13 similar to the image processing according to the first embodiment are performed without performing Step S9 similar to the image processing according to the first embodiment. Then, when it has a sufficient value (the number of current images) in the determination of step S11 (step S11; YES), the operation of step S9 similar to the image processing according to the first embodiment is performed, and thereafter The operation of step S14 similar to the image processing according to the first embodiment is executed.

  The image processing according to the second embodiment described above can achieve the same effects as the image processing according to the first embodiment described above.

In addition, when performing guidance information generation processing in the image processing according to the first embodiment or the image processing according to the second embodiment, the following data may be cited as the basic data other than the resolution described above.
Data indicating the accuracy of the alignment processing according to steps S4 to S7 (the CPU 1 can generate guide information so that an area with lower accuracy is imaged more carefully)
Data indicating the amount of deviation related to the result of the alignment processing according to steps S4 to S7, or data indicating the cause of the deviation (the CPU 1 considers the cause so that the amount of deviation decreases). Guide information can be generated.)
Data indicating the movement path or movement speed of the smartphone S moved in the alignment process according to steps S4 to S7 (CPU 1 is more regular when the movement path is too irregular or the movement speed is too fast. Can guide route and slower movement.)
-The number of current images GT imaged immediately before in the alignment processing according to steps S4 to S7 and the time required to capture the number of current images GT (the CPU 1 sets a region with lower accuracy to a longer time (Guide information can be generated so as to capture images over time.)
Of the current images GT captured immediately before in the alignment processing according to the above steps S4 to S7, the ratio of the current image GT having a satisfactory alignment result to the entire captured current image GT (the CPU 1 determines the ratio Guide information can be generated so that the
(III) Modifications Note that the present invention can be applied in various ways other than the embodiments described above.

  First, as a first modification, for the guidance information to be displayed on the display 5 (see FIG. 2 or step S13 in FIG. 7), the desired movement of the smartphone S is not displayed instead of displaying the difference in resolution for each area by color. The direction may be displayed on the display 5 such as “up”, “slower”, “still”. Moreover, you may comprise so that these guidance may be sounded through the speaker 7 with an audio | voice. In these cases, the CPU 1 displays guidance information on the content of moving the smartphone S on the display 5 in order to capture an area of a smaller data amount based on the data amount acquired in the process of step S12. According to this 1st modification, since guidance information is the guidance information which shows the item (how to move) of the movement of the smart phone S, the smart phone S can be moved easily. Furthermore, as a further modification of the guidance information, it corresponds to the difference in resolution for each area, for example, the difference in visual hatching pattern, the difference in transparency (assuming that the background exists), and the difference in the resolution. It is also possible to display by using a visualization method in which different display modes are associated with the blinking (flashing) of the time interval, that is, the difference in resolution for each region.

  Next, as a second variation, image processing in the smartphone S is interrupted after completion of at least one of the alignment processing and the synthesis processing according to each embodiment, and corresponds to the result of the completed alignment processing or synthesis processing The result data can be stored in, for example, the RAM 3 or a hard disk (not shown). In this case, it is preferable that the CPU 1 obtains the stored result data again and uses the result data to restart the interrupted process. In the case of this second modification, even if it is necessary to interrupt the image processing in the smartphone S after at least one of the alignment process and the composition process is completed, the image is interrupted using the stored result data. The process can be resumed, and the degree of freedom in executing the alignment process and the synthesis process can be improved. Further, in the second modification, the CPU 1 includes image data corresponding to the result of the alignment process or the synthesis process up to the interruption in the result data, and the result data is stored in the image recognition device outside the smartphone S. It can also be configured to output to In this case, if the image recognition apparatus has an image recognition capability higher than that of the CPU 1, for example, the recognition result is acquired as result data used at the time of restart, so that the composite image as the smartphone S has higher image quality or It can be synthesized over a wide range. That is, for example, when the user temporarily interrupts the image processing with his / her intention (for example, when the user moves the tertile smartphone S to perform image processing and then temporarily stops), in the external image recognition device The image processing result up to the interruption is analyzed, and the analysis result is acquired to generate guidance information for identifying a portion that requires further fine imaging. Then, when the wearer further continues to capture the portion identified by the guidance information, a synthesized image having a higher image quality or a wider range can be synthesized.

  As a third variation, the smartphone S according to the first embodiment or the second embodiment is provided with a light as an example of an “illuminating unit” that illuminates a part or all of the imaging object during imaging by the camera 9. The illuminance of the light may be controlled by the CPU 1 based on at least the result of the alignment process. In the case of the third modification, at least a part or the whole of the imaging object is illuminated using the result of the positioning process, so that a higher quality composite image can be synthesized.

  Further, as a fourth modification, the CPU 1 performs the alignment process using image data output from a plurality of cameras other than the camera 9 and having different relative positions to the imaging target. It may be configured. In the case of the fourth modification, the alignment process is performed using image data output from each of a plurality of other cameras having different relative positions with respect to the imaging target. It is possible to synthesize a composite image having a higher image quality or a wider range. In addition, the other camera in this case may be another camera provided in the smartphone S other than the camera 9, or may be a camera mounted on another portable information terminal or the like. Furthermore, for a three-dimensional object, the current image GT corresponding to a plurality of viewpoints is captured by using a position sensor provided in the smartphone S and a plurality of buffers in the RAM 3, thereby increasing the height of the object. It can be digitized in image quality or in a wide range.

  Furthermore, as a fifth variation, data indicating the position or range in the imaging target occupied by a part of the imaging target included in the image after the alignment process (that is, in the entire imaging target) The CPU 1 can also be configured to generate guidance information without using the pixel values or the like based on data indicating which part of the current image GT is captured). Even in the case of the fifth modification, the smartphone S to be carried can be moved accurately and quickly to a position necessary for capturing an image necessary for synthesizing a high-quality image or a wide range of synthesized images. .

  Further, as a sixth modification, the CPU 1 outputs composite image data corresponding to the composite image synthesized to an external image quality evaluation device (for example, an external OCR (Optical Character Leader) device), and further uses the read result. In the case of the sixth modification, the guide information is generated based on the image quality of the combined image that has been combined (that is, the read result). A high-quality image or a wide range of synthesized images can be synthesized thereafter.

  Finally, as a seventh modified form, the CPU 1 acquires specifications such as the imaging specifications (shutter speed, aperture, focal length, amount of light) in the camera 9 or the position, speed, and acceleration of the smartphone S itself, and based on these. The CPU 1 may be configured to select the current image GT or the entire image GA that is the object of the alignment process. Specifically, the CPU 1 selects a current image GT or the like that is favorable as an imaging condition (in other words, already has high image quality by itself) based on the above specifications and the like, and uses the current image GT for the subsequent alignment processing. In this case, since the original current image GT or the like is considered to have a high image quality, a desired high image quality or a wide range of synthesized images can be synthesized more quickly using a small number of current images GT or the like. become.

  The program corresponding to the flowchart shown in FIG. 2 or FIG. 7 is acquired via a network such as the Internet, or the program recorded on an information recording medium such as an optical disk is acquired, for example, by a general-purpose microcomputer. It can also be configured to read and execute this. The microcomputer in this case executes the same processing as the CPU 1 according to each embodiment.

  As described above, the present invention can be used in the field of information processing apparatuses, and particularly when applied to the field of information processing apparatuses for image processing, a particularly remarkable effect can be obtained. In addition, as described above, anytime, anywhere, anywhere, personal privacy and work-related information security at the highest level, and easy high-quality or wide-range image synthesis without the hassle. It can be carried out. This is an epoch-making value that was not available in previous devices.

1 CPU
2 ROM
3 RAM
DESCRIPTION OF SYMBOLS 4 Operation part 5 Display 6 Call control part 7 Speaker 8 Microphone 9 Camera 10 Communication interface 30 Alignment data buffer 31 Data amount buffer 32 Composite image buffer 33 Wide area image buffer 34 Current image buffer 35 Aligned current image buffer 4A, 4B, 4C Operation button S Smartphone ANT Antenna GA Overall image GT Current image GTa, GTb, GTc, GTd Split image

Claims (21)

In a portable information processing device,
Imaging means for imaging at least a part of the imaging object and outputting frame image information corresponding to a frame image corresponding to the captured at least part;
A plurality of the frames corresponding to the respective frame images are subjected to an alignment process for aligning a plurality of the frame images continuously captured by the imaging means as the portable information processing device moves. Using image information, positioning means that sequentially performs corresponding to continuous imaging of the frame image;
Using the frame image information corresponding to each of the aligned frame images, the composite image corresponding to the imaging target and having a higher image quality or a wider range of synthesis than each frame image captured by the imaging unit A combining means for performing a combining process for combining images;
An information processing apparatus comprising: The information processing apparatus according to claim 1,
The alignment means includes whole frame image information corresponding to a whole frame image corresponding to the whole of the imaging object, and a plurality of pieces of frame image information corresponding to the frame images respectively corresponding to the parts. And an information processing apparatus that sequentially performs the alignment processing corresponding to the continuous imaging. The information processing apparatus according to claim 1 or 2,
The information processing apparatus, wherein the alignment means sequentially performs the non-rigid alignment process using a plurality of pieces of frame image information. In the information processing apparatus according to any one of claims 1 to 3,
The part of the imaging target to be continuously captured based on the captured position range information indicating the position or range in the imaging target occupied by the part corresponding to the frame image after the alignment process Position range information generating means for generating position range information indicating a position or range on an object;
Guidance information generating means for generating guidance information for guiding the movement of the information processing apparatus to a position corresponding to the position range information based on the generated position range information;
Notification means for notifying the generated information to the outside of the information processing device;
An information processing apparatus comprising: In the information processing apparatus according to any one of claims 1 to 3,
Position range information generating means for generating position range information indicating the position or range of the part of the imaging target to be continuously captured based on the captured frame image;
Guidance information generating means for generating guidance information for guiding the movement of the information processing apparatus to a position corresponding to the position range information based on the generated position range information;
Notification means for notifying the generated information to the outside of the information processing device;
An information processing apparatus comprising: In the information processing apparatus according to claim 4 or 5,
The synthesizing unit sequentially performs the synthesis process based on the frame image information corresponding to the captured frame image corresponding to continuous imaging of the frame image,
The position range information generating means includes
An external output means for outputting composite image information corresponding to the synthesized composite image to an external image quality evaluation means;
Acquisition means for acquiring evaluation result information indicating an evaluation result by the image quality evaluation means for the output composite image information;
With
An information processing apparatus that generates the position range information by further using the acquired evaluation result information. The information processing apparatus according to claim 5,
The position range information generation unit generates the position range information based on the frame image after the alignment process. In the information processing apparatus according to claim 4 or 7,
The position range information generation unit includes an accumulation unit that accumulates at least the number of the frame images used in the alignment process for each pixel of the frame image after the alignment process,
The information processing apparatus according to claim 1, wherein the position range information generation unit generates the position range information based on the accumulated number. In the information processing apparatus according to any one of claims 4 to 8,
The guidance information is guidance information visually recognized by a carrier of the information processing device,
An information processing apparatus further comprising display means for displaying the guidance information. The information processing apparatus according to claim 9,
The information processing apparatus according to claim 1, wherein the guidance information is guidance information displayed in an aspect in which the image quality of the synthesized image after the synthesis process can be identified for each image quality. In the information processing apparatus according to claim 9 or 10,
The information processing apparatus, wherein the guide information is guide information displayed in a color corresponding to an image quality in the composite image after the composite processing. The information processing apparatus according to any one of claims 9 to 11,
The information processing apparatus characterized in that the display means displays the frame image and displays the guidance information so as to overlap the displayed frame image. The information processing apparatus according to any one of claims 9 to 11,
The information processing apparatus characterized in that the display means displays a whole frame image corresponding to the whole of the imaging object, and displays the guidance information superimposed on the whole frame image. The information processing apparatus according to any one of claims 9 to 13,
The information processing apparatus is characterized in that the guide information is guide information indicating specifications of movement of the portable information processing apparatus. The information processing apparatus according to any one of claims 1 to 14,
Interruption means for interrupting the processing in the information processing apparatus after the end of at least one of the alignment processing or the synthesis processing;
Storage means for storing result information corresponding to at least one of the completed processing results;
Result information acquisition means for acquiring stored result information;
Resuming means for resuming the suspended processing using the acquired result information;
An information processing apparatus further comprising: The information processing apparatus according to claim 15,
The result information includes frame image information corresponding to the at least one result up to the interruption,
The storage means includes external output means for outputting the result information to an external image recognition means,
The information processing apparatus, wherein the result information acquisition unit acquires the result information including frame image information corresponding to a frame image recognized by the image recognition unit. The information processing apparatus according to any one of claims 1 to 16,
Illuminating means for illuminating a part or all of the imaging object during imaging by the imaging means;
Illumination control means for controlling the illumination means based at least on the result of the alignment process;
An information processing apparatus comprising: The information processing apparatus according to any one of claims 1 to 17,
The positioning unit performs the positioning process using the frame image information respectively output from the plurality of imaging units having different relative positions with respect to the imaging object. apparatus. Executed in an information processing apparatus that is portable and includes an imaging unit that captures at least a part of an imaging target and outputs frame image information corresponding to a frame image corresponding to the captured at least part Information processing method,
A plurality of the frames corresponding to the respective frame images are subjected to an alignment process for aligning a plurality of the frame images continuously captured by the imaging means as the portable information processing device moves. Using image information, an alignment step that is sequentially performed corresponding to continuous imaging of the frame image;
Using the frame image information corresponding to each of the aligned frame images, the composite image corresponding to the imaging target and having a higher image quality or a wider range of synthesis than each frame image captured by the imaging unit A compositing process for compositing an image;
An information processing method comprising: Included in an information processing apparatus that is portable and includes an imaging unit that captures at least a part of an imaging target and outputs frame image information corresponding to a frame image corresponding to the captured at least part Computer
A plurality of the frames corresponding to the respective frame images are subjected to an alignment process for aligning a plurality of the frame images continuously captured by the imaging means as the portable information processing device moves. Using the image information, alignment means that sequentially performs corresponding to continuous imaging of the frame image, and
Using the frame image information corresponding to each of the aligned frame images, the composite image corresponding to the imaging target and having a higher image quality or a wider range of synthesis than each frame image captured by the imaging unit A combining means for performing a combining process for combining images;
An information processing program characterized by functioning as Included in an information processing apparatus that is portable and includes an imaging unit that captures at least a part of an imaging target and outputs frame image information corresponding to a frame image corresponding to the captured at least part Computer
A plurality of the frames corresponding to the respective frame images are subjected to an alignment process for aligning a plurality of the frame images continuously captured by the imaging means as the portable information processing device moves. Using the image information, alignment means that sequentially performs corresponding to continuous imaging of the frame image, and
Using the frame image information corresponding to each of the aligned frame images, the composite image corresponding to the imaging target and having a higher image quality or a wider range of synthesis than each frame image captured by the imaging unit A combining means for performing a combining process for combining images;
An information recording medium on which an information processing program that functions as a computer is recorded so as to be readable by the computer.