JP2011029881A - Portable information processing apparatus, portable information processing method and portable information processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To further improve operability, in a portable information processing apparatus which can be carried for performing information processing that uses images. <P>SOLUTION: A reference state photographic means photographs a plurality of reference states of an object to obtain a reference image. An object photography means photographs states of the object. A comparison means compares an image photographed by the object photographic means with the reference image photographed by the reference state photographic means to generate comparison information. A control means controls information processing, on the basis of the comparison information generated by the comparison means. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a portable portable information processing apparatus, a portable information processing method, and a portable information processing program that perform information processing using images.

  In recent years, studies on techniques for recognizing an object or a change thereof using a photographed image have been underway. As a technique related to this, for example, Patent Document 1 discloses a motion detection device that can accurately detect a motion regardless of the position of the detected object and has a wide application range of the detected object. Specifically, first, based on the image data, an area where the magnitude of the movement vector is larger than a predetermined value is extracted as a vector area where the movement vectors are continuous. Then, based on the movement vector corresponding to the vector area, a motion pattern corresponding to the motion state of the subject is detected. Further, the detected motion pattern is compared with each of a plurality of registered patterns stored in the registered pattern storage unit, and a registered pattern closest to the motion pattern is extracted.

  Further, for example, in Patent Document 2, even if the imaging direction of the surveillance camera changes, it is detected that the imaging position of the surveillance camera has changed in a short time from the difference in the amount of change in luminance between the reference line and the feature point. A surveillance camera system that informs a supervisor of an abnormality is disclosed. When at least one reference pixel is selected in accordance with the operation of the operator, the surveillance camera system stores position information of the reference pixel and each pixel that is set as the reference pixel in the image captured by the imaging unit Memorize the brightness. When a new image is picked up by the image pickup means, a pixel corresponding to the position information of the reference pixel is extracted from the picked up image, and a luminance difference for each pixel corresponding to the extracted pixel and the reference pixel is extracted. Ask for. If the luminance difference between the two is greater than or equal to a predetermined value, it is determined that the imaging direction of the imaging means has changed.

Japanese Patent Laid-Open No. 2002-024835 JP 2001-359086 A

However, such a conventional technique can recognize an object or a change thereof using a photographed image, but using an image to control information processing by a portable information processing device such as a cellular phone is not possible. Not done. In particular, with the development of portable information processing devices and their communication services, information processing itself such as games can be easily performed, but generally portable information processing devices have a small casing and a limited number of keys. Has been. Therefore, input as a user interface is difficult depending on the information processing to be executed.
The present invention has been made paying attention to such problems of the prior art, and aims to further improve the operability in a portable information processing apparatus that performs information processing using images. Yes.

The gist of the present invention for achieving the object lies in the inventions of the following items.
[1] A portable portable information processing apparatus that performs information processing using an image. This portable information processing apparatus includes a reference state photographing unit, an object photographing unit, a comparison unit, and a control unit. The reference state photographing unit obtains a reference image by photographing a plurality of reference states of the object. The object photographing means photographs the state of the object. The comparison unit generates comparison information by comparing the image photographed by the object photographing unit and the reference image photographed by the reference state photographing unit. The control means controls information processing based on the comparison information generated by the comparison means.

  [2] In addition, it is preferable that this portable information processing apparatus further includes a state detection unit that detects a state of the portable information processing device when shooting is performed by the object shooting unit. In this case, the comparison unit corrects the image photographed by the object photographing unit based on the state detected by the state detection unit.

  [3] Further, when the object is not photographed by the object photographing means, the control means controls information processing based on comparison information generated by the comparing means for the object photographed in the past. Is preferred.

  [4] Furthermore, the portable information processing device includes a transmission unit that transmits a reference image captured by the reference state imaging unit or an image captured by the object imaging unit to the information processing device, and a reference image from the information processing device. It is preferable to further include receiving means for receiving comparison information with an image. In that case, the control means controls information processing based on the comparison information received by the receiving means.

  [5] A portable information processing method performed by a portable portable information processing apparatus that performs information processing using an image. The portable information processing apparatus includes a reference state photographing unit, an object photographing unit, a comparison unit, and a control unit. The reference state photographing unit obtains a reference image by photographing a plurality of reference states of the object. The object photographing means photographs the state of the object. The comparison unit generates comparison information by comparing the image photographed by the object photographing unit and the reference image photographed by the reference state photographing unit. The control means controls information processing based on the comparison information generated by the comparison means.

  [6] A portable information processing program that allows a portable portable information processing device incorporating a computer to perform information processing using an image. Make the computer function as follows. The reference state photographing unit obtains a reference image by photographing a plurality of reference states of the object. The object photographing means photographs the state of the object. The comparison unit generates comparison information by comparing the image photographed by the object photographing unit and the reference image photographed by the reference state photographing unit. The control means controls information processing based on the comparison information generated by the comparison means.

  According to the portable information processing device, the portable information processing method, and the portable information processing program according to the present invention, a portable portable information processing device that performs information processing using an image, photographs a state of an object, Information processing can be controlled by comparison with an image of a standard state of an object. And the operativity in the portable information processing apparatus is further improved.

It is a conceptual module block diagram about the structural example of this Embodiment. It is a flowchart which shows the process example of the preparation stage by a portable information processing apparatus. It is explanatory drawing which shows the example of a reference | standard image. It is a flowchart which shows the game processing example by a portable information processing apparatus. It is explanatory drawing which shows the example of a target image. It is explanatory drawing which shows the example of the target image inclined. It is explanatory drawing which shows the example when the target object is not fully image | photographed in the target image. It is explanatory drawing which shows the system configuration example in the case of implement | achieving 2nd Embodiment. It is a flowchart which shows the example of a game process by an image processing server. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

<First Embodiment>
Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment (first embodiment, second embodiment).
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment also serves as an explanation of a computer program, a system, and a method. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used, but these words are stored in a storage device when the present embodiment is a computer program, or It means to control to store in the storage device. In addition, the modules correspond almost one-to-one with the functions. However, in mounting, one module may be composed of one program, or a plurality of modules may be composed of one program. A plurality of programs may be used. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. Hereinafter, “connection” is used not only for physical connection but also for logical connection (data exchange, instruction, reference relationship between data, etc.).
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is included. “Apparatus” and “system” are used as synonymous terms. The term “predetermined” is used in addition to a predetermined meaning, including the meaning according to the situation / state at that time or the situation / state until then.

  Hereinafter, a mobile phone will be mainly exemplified and described as the portable information processing apparatus. In addition, a game is mainly exemplified and described as information processing performed by the portable information processing apparatus.

  A portable information processing apparatus 100 according to the present embodiment is intended to input information for controlling information processing using a camera built in the portable information processing apparatus, as shown in FIG. In addition, the camera 110, the sensor 115, the communication module 120, the display 125, the input module 130, and the processing module 150 are included. The processing module 150 includes a control module 155, a memory 160, a feature extraction module 165, a comparison module 170, and an information processing module 175. In this embodiment, the camera 110 is used as the reference state photographing means and the object photographing means described in the claims, the comparison module 170 is used as the comparison means, the control module 155 is used as the control means, and the sensor 115 is used as the state detection means. The communication module 120 is shown as an example as a transmission unit and a reception unit, respectively.

  The camera 110, the sensor 115, the communication module 120, the display 125, the input module 130, the memory 160, the feature extraction module 165, the comparison module 170, and the information processing module 175 are connected to the control module 155 and controlled by the control module 155. Is done.

  The camera 110 is a camera built in the portable information processing apparatus 100. According to a user operation, the camera 110 captures a plurality of reference states of an object to obtain a reference image, and the reference image is used as a control module 155. To pass. Also, the state of the object is photographed to obtain a target image, and the target image is transferred to the control module 155. The reference image is for generating the maximum value or the minimum value of the control value necessary for controlling the information processing module 175, and the state of the object when the control value of the maximum value or the minimum value is input Was taken. This is necessary in advance for the information processing module 175 to perform information processing. The target image is used to generate a control value necessary for controlling the information processing module 175. When the control value is input while the information processing module 175 is performing information processing, the target image is displayed. This is a photograph of the state. Further, the object is not limited, but is easy for the user to operate, and is easy to shoot with the camera 110. In particular, the whole is one color, and it is good that the color is different from the background. For example, a user's foot (shoes) corresponds. The lens of the camera 110 may be anywhere in the main body of the portable information processing apparatus 100, but it is suitable that the lens is on the back side of the display 201 in order to shoot while checking the state image of the object on the display 201. .

  The sensor 115 is built in the portable information processing apparatus 100 and detects the state of the portable information processing apparatus 100 when shooting is performed by the camera 110. For example, specifically, there are an acceleration sensor, a geomagnetic sensor, an angular velocity sensor, and the like, and in particular, any device that can detect the tilt of the camera 110 may be used. Further, the detection timing is when the camera 110 takes a picture, and this timing is controlled by the control module 155.

The communication module 120 communicates with other devices. Specifically, an antenna, a communication control circuit, etc. correspond. In particular, the reference image or target image captured by the camera 110 is transmitted to the image processing server 850. Then, the comparison information between the reference image and the target image is received from the image processing server 850.
The display 125 is provided in the portable information processing apparatus 100, and specifically corresponds to a liquid crystal screen, a display control circuit, and the like. Images taken by the camera 110, processing results by the information processing module 175, and the like are displayed under the control of the control module 155.
The input module 130 is provided in the portable information processing apparatus 100, inputs a user operation, and passes the operation information to the control module 155. For example, specifically, there are a key, a touch panel integrated with the display 125, and the like, which have a function as a shutter button.

The memory 160 stores an image photographed by the camera 110, feature values extracted by the feature extraction module 165 for the image, comparison information generated by the comparison module 170, arguments necessary for processing of the information processing module 175, and the like. To do.
The feature extraction module 165 extracts features of an image taken by the camera 110. Specific features to be extracted include, for example, the area of the object, the distribution of the object in the image, the inclination of the center line of the object, the shape of the object, and the like. These extractions are performed using existing image processing.

  The comparison module 170 compares the target image captured by the portable information processing apparatus 100 with the reference image and generates comparison information. Further, the comparison module 170 may correct the target image (including the feature value of the target object in the target image) captured by the camera 110 based on the state detected by the sensor 115. Good. Further, the comparison module 170 compares at least the color information of the object in the image as the object to be compared. In addition, the area of the object, the distribution of the object in the image, the inclination of the center line of the object, the shape of the object, and the like may be compared. In these cases, the color information of the object is prioritized (compared first, or the weight of the maximum value is given when generating the comparison information, etc.) for comparison. For comparison between images, the feature value extracted by the feature extraction module 165 is used. The comparison information includes the similarity with the reference image, the inclination of the object, and the like. Note that the tilt of the target object is the tilt of the target object in the first reference image (A1), the tilt of the target object in the second reference image (A2), and the tilt of the target object in the target image (A3). Is a value indicating which A3 is inclined in the angle formed by A1 and A2. More specifically, it is calculated by | A3-A1 | / | A2-A1 |.

  The information processing module 175 performs information processing. The information processing here is not particularly limited, but corresponds to a game or the like. For example, a character in the game is operated by changing an object.

  The control module 155 controls each module (including the camera 110 and the like). In particular, the information processing module 175 is controlled based on the comparison information generated by the comparison module 170. The control module 155 controls the information processing module 175 based on the comparison information generated by the comparison module 170 for the object photographed in the past when the object is not photographed by the camera 110. May be. Further, the control module 155 may control the information processing module 175 based on the comparison information received by the communication module 120. Note that the comparison information generated by the comparison module 170 is control information for the information processing module 175.

FIG. 2 is a flowchart illustrating an example of a preparation stage process performed by the portable information processing apparatus 100. This is processing that is performed before full-scale processing by the information processing module 175 is performed.
In step S202, the camera 110 captures the first reference image. For example, the first reference image is the reference image a300 illustrated in FIG. This is a photograph of a foot (shoes) that is an object 310 tilted to the right. For example, it indicates an instruction to advance a character at the highest speed in a game performed by the information processing module 175. Further, in order to clarify the target object 310, the target object 310 may be selected in accordance with a user operation.
In step S <b> 204, the control module 155 stores the first reference image in the memory 160.

In step S206, the camera 110 captures the second reference image. For example, the second reference image is the reference image b350 illustrated in FIG. This is a photograph of a foot (shoes) that is an object 360 tilted to the left, and indicates, for example, an instruction to reverse the character at the highest speed in a game performed by the information processing module 175. Further, in order to clarify the object 360, the object 360 may be selected in accordance with the user's operation.
In step S <b> 208, the control module 155 stores the second reference image in the memory 160.

  In step S210, the feature extraction module 165 extracts the feature of the object in the reference image. Here, for example, the feature to be extracted is the angle of the center line of the object 310 (360). That is, it is possible to instruct forward, backward, and speed according to the angle of the foot (shoe).

In step S212, the feature extraction module 165 stores the feature value in the memory 160. For example, a reference image feature table is stored. The reference image feature table has a color information column, an area column, a distribution column, a centerline inclination column, a shape column, and the like in the column direction, and a first reference image column and a second reference image column in the row direction. have. That is, the feature value of each reference image is stored. Note that the reference image feature table may store only necessary feature values.
The color information is RGB pixel values of the captured object 310 (360), the area is the number of pixels of the object 310 (360), the distribution is the center (center of gravity) of the object 310 (360), and the like. Are the coordinates in the reference image a300 (reference image b350), the inclination of the center line is the inclination angle of the center line of the object 310 (360), and the shape is a feature representing the shape of the object 310 (360). A value (for example, a circle having a radius from the center to the farthest point is drawn, and the ratio of the object 310 (360) in the circle, that is, a characteristic value indicating the circularity).

FIG. 4 is a flowchart showing an example of game processing by the portable information processing apparatus 100.
In step S402, the control module 155 causes the information processing module 175 to start a game in response to a user operation on the input module 130.
In step S404, the camera 110 captures the target image. For example, the target image is the target image 500 illustrated in FIG. This is a photograph of a foot (shoes) that is the object 510 tilted slightly to the right, and indicates, for example, an instruction to move the character forward at a slightly higher speed in a game played by the information processing module 175. Further, in order to clarify the target object 510, the target object 510 may be selected in accordance with a user operation.

In step S406, the sensor 115 detects the state of the portable information processing apparatus 100. That is, the state (tilt or the like) of the portable information processing apparatus 100 at the time of shooting in step S404 is detected.
In step S408, image features are extracted. The same processing as step S210 in the flowchart illustrated in FIG. 2 is performed. The feature extraction result is stored in the target image feature table. The target image feature table has a color information column, an area column, a distribution column, a center line inclination column, a shape column, and the like in the column direction, and has a target image column in the row direction. That is, it is equivalent to the reference image feature table in the reference image. There may be a plurality of objects.

  In step S410, the control module 155 determines whether or not there is an object in the target image as a result of the process in step S408. If there is, the process proceeds to step S412; otherwise, the process proceeds to step S418. That is, it is determined whether or not an object similar to the object 310 (360) in the reference image a300 (reference image b350) is in the photographed target image. It is determined whether or not the area of the color information of the target object in the image and the area of the color information of the target object in the target image captured this time are within a predetermined range. FIG. 7 is an explanatory diagram illustrating an example where the object 710 is not sufficiently captured in the object image 700. Although the target object 710 is photographed in the target image 700, since the area is not sufficient, it is determined that the target object is not photographed.

In step S412, the comparison module 170 corrects the feature value of the object based on the value detected in step S406. For example, when it is detected in step S406 that the portable information processing apparatus 100 is tilted, the photographed target image is also tilted, so that the feature value (for example, the tilt of the center line) of the target is corrected.
FIG. 6 is an explanatory diagram illustrating an example of a tilted target image. The target image 600 is obtained by photographing the target object 610 by tilting the portable information processing apparatus 100 to the left (in FIG. 6, the vertical line 620 is tilted to the left). Accordingly, the inclination of the center line is corrected to return to the right (the vertical line 620 is returned to the original vertical). In the example of FIG. 6, if this correction is not performed, it is interpreted as an instruction for moving forward at a low speed, but it can be interpreted as an instruction for moving forward at a slightly higher speed by correcting the inclination. For example, such a case occurs in a game in which the entire portable information processing apparatus 100 is used as a handle, and even in such a case, an accurate instruction can be given by an object.

  In step S414, the comparison module 170 compares the feature value of the reference image with the game input value, which is comparison information (control value). That is, the value in the target image feature table that stores the feature value of the target object in the target image (here, the inclination of the center line) and the value in the reference image feature table that stores the feature value of the target object in the reference image And how much the object in the target image is similar to the object in the reference image a or the reference image b is calculated. In the example of FIG. 6, the object is similar to the object in the reference image a, and the similarity is an angle difference (in this case, the smaller the value, the more similar).

In step S416, the control module 155 stores the game input value in the memory 160. This is to cope with a case where the object cannot be detected. Next, the process proceeds to step S420.
In step S418, the control module 155 extracts the previous game input value from the memory 160. That is, the game input value stored in step S416 is extracted.
In step S <b> 420, the control module 155 inputs a game input value to the information processing module 175. As the game input value, in the example of FIGS. 5 and 6, an instruction to advance the character at a slightly higher speed is input.

<Second Embodiment>
The first embodiment described above is an example of processing when the portable information processing apparatus 100 is used as a stand-alone, but can also be used as a client-server type.
FIG. 8 is an explanatory diagram showing an example of a system configuration when the second embodiment is realized. The mobile phone 800 and the image processing server 850 are connected via a wireless or wired communication network 890. A plurality of mobile phones 800 and image processing servers 850 may be provided. However, preparatory stage processing and game processing, which will be described later, are performed between the pair of mobile phones 800 and the image processing server 850.
The mobile phone 800 is a specific example of the mobile information processing apparatus 100 shown in the example of FIG.
The image processing server 850 mainly performs processing of the feature extraction module 165 and the comparison module 170 shown in the example of FIG. 1, receives an image from the mobile phone 800, extracts the feature of the image, and generates comparison information. Then, the data is transmitted to the mobile phone 800. That is, the mobile phone 800 performs processing that is difficult in terms of performance.

Next, a processing example in the system configuration illustrated in FIG. 8 will be described.
First, an example of processing at the preparation stage by the mobile phone 800 will be described.
(1) The processing from step S202 to step S208 in the flowchart illustrated in FIG. 2 is performed.
(2) The communication module 120 transmits the first reference image and the second reference image in the memory 160 to the image processing server 850.

Next, an example of processing at the preparation stage by the image processing server 850 will be described.
(1) The first reference image and the second reference image transmitted from the mobile phone 800 are received.
(2) Extract features of the object in the reference image. This corresponds to the process of step S210 in the flowchart illustrated in FIG.
(3) Mobile phone number that has been sent (“Mobile phone number” is a unique identification number that uniquely identifies the mobile phone, and other codes that uniquely identify the mobile phone. In the following description, the feature values are stored in association with each other. That is, it corresponds to a case where a plurality of mobile phones 800 transmit data to one image processing server 850. Specifically, a cell phone number column may be provided in the reference image feature table.

Next, an example of game processing by the mobile phone 800 will be described.
(1) The processing from step S402 to step S406 in the flowchart illustrated in FIG. 4 is performed.
(2) The communication module 120 transmits the captured image and the sensor detection value to the image processing server 850.
(3) The communication module 120 receives a game input value from the image processing server 850.
(4) The control module 155 inputs a game input value to the information processing module 175. This corresponds to the process of step S420 in the flowchart illustrated in FIG.

FIG. 9 is a flowchart showing an example of game processing by the image processing server 850.
In step S902, the target image and the sensor detection value transmitted from the mobile phone 800 are received.
The processing from step S904 to step S914 corresponds to the processing from step S408 to step S418 in the flowchart illustrated in FIG. However, the processing in step S910, step S912, and step S914 is performed in accordance with the mobile phone number of the mobile phone 800 that has transmitted.

That is, in step S910, it is compared with the feature value of the reference image corresponding to the transmitted mobile phone number, and converted into a game input value. That is, it is compared with the feature value of the object in the reference image corresponding to the mobile phone 800 transmitted this time. Specifically, a feature value corresponding to the mobile phone number of the mobile phone 800 transmitted this time is extracted from the reference image feature table and compared.
In step S912, the game input value is stored in association with the transmitted mobile phone number. This is to cope with a case where the target object of the target image transmitted next time from the mobile phone 800 cannot be detected.
In step S914, the previous game input value corresponding to the transmitted mobile phone number is extracted. That is, the game input value stored in step S912 is extracted.
In step S916, the game input value is transmitted to mobile phone 800.

  Note that the hardware configurations of the mobile phone 800 and the image processing server 850 that execute the program according to the present embodiment are general computers as shown in FIG. A CPU 1010 that executes programs such as a control module 155, a feature extraction module 165, a comparison module 170, and an information processing module 175, a RAM 1030 that stores the programs and data, a program for starting the computer, and the like are stored. The ROM 1020, the HD 1040 as an auxiliary storage device (not the mobile phone 800), the data input by the operator's operation from the key, the mouse, etc., or the data is output to the CRT, liquid crystal display, etc. An interface UI / F 1050, a removable media reader / writer 1060 for reading from and writing to a removable medium such as a CD-R or a memory card, a communication line I / F 1070 for connecting to a communication network, and these Ide and a bus 1080 for exchanging data. A plurality of these computers may be connected to each other via a network.

In the embodiment, the game is shown as information processing performed by the mobile information processing apparatus 100 (the mobile phone 800 in the second embodiment), but it may be a browser operation or the like. Of course, the object is something other than a foot (shoe) (for example, a hand opposite to the operating hand, a poster with a plurality of types of figures (objects) pasted on the wall, etc.) Also good. Further, as a feature to be compared, in addition to the inclination of the center line, the area, the position (distribution) in the image, and the like may be used.
In addition, the mobile information processing apparatus has been described mainly by taking a mobile phone as an example, but the first embodiment may not be communicable as long as it is an information processing apparatus that is portable and includes a photographing device. (Of course, communication may be possible). For example, it may be a game machine, a PDA (Personal Digital Assistant), an electronic dictionary, a portable navigation machine, or the like. The second embodiment may be a PHS (Personal Handyphone System) or the like in addition to the above-described game machine capable of communication.

Further, in the description of the processing example in the preparation stage described above, the feature value of the object is not corrected, but also in the preparation stage processing, based on the state of the portable information processing device 100 detected by the sensor 115. Thus, the feature value of the object in the reference image may be corrected.
Note that the technology described in the background art may be adopted as the processing content of each module.
In the above-described embodiment, the image obtained by capturing the state of the object when the maximum or minimum control value is input is illustrated as the reference image. However, an image corresponding to the middle (for example, a neutral value is shown). Or an image obtained by photographing the state of the object. That is, the reference image may be three or more. If there is an image corresponding to the middle, there is an effect that accuracy can be improved when a control value is input.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.

DESCRIPTION OF SYMBOLS 100 ... Portable information processing apparatus 110 ... Camera 115 ... Sensor 120 ... Communication module 125 ... Display 130 ... Input module 150 ... Processing module 155 ... Control module 160 ... Memory 165 ... Feature extraction module 170 ... Comparison module 175 ... Information processing module 800 ... Mobile phone 850 ... Image processing server 890 ... Communication network

Claims (6)

A portable information processing device that performs information processing using images,
A reference state photographing means for photographing a plurality of reference states of an object to obtain a reference image;
An object photographing means for photographing the state of the object;
Comparison means for comparing the image photographed by the object photographing means with the reference image photographed by the reference state photographing means to generate comparison information;
A portable information processing apparatus comprising: control means for controlling the information processing based on comparison information generated by the comparison means. A state detecting means for detecting the state of the portable information processing apparatus when the object photographing means is photographed;
The portable information processing apparatus according to claim 1, wherein the comparison unit corrects an image photographed by the object photographing unit based on a state detected by the state detection unit. The control means controls the information processing based on comparison information generated by the comparison means for an object photographed in the past when the object photographing means has not been photographed by the object photographing means. The portable information processing apparatus according to claim 1, wherein the portable information processing apparatus is a portable information processing apparatus. A transmission means for transmitting a reference image photographed by the reference state photographing means or an image photographed by the object photographing means to an information processing device;
Receiving means for receiving comparison information between the reference image and the image from the information processing apparatus;
The portable information processing apparatus according to any one of claims 1 to 3, wherein the control unit controls the information processing based on comparison information received by the receiving unit. A portable information processing method performed by a portable information processing apparatus that performs information processing using an image,
The reference state photographing means obtains a reference image by photographing a plurality of reference states of the object,
The object photographing means photographs the state of the object,
The comparing means generates comparison information by comparing the image photographed by the object photographing means and the reference image photographed by the reference state photographing means,
The portable information processing method, wherein the control means controls the information processing based on the comparison information generated by the comparison means. A portable information processing program capable of causing a portable portable information processing apparatus incorporating a computer to perform information processing using an image,
The computer,
A reference state photographing means for photographing a plurality of reference states of an object to obtain a reference image;
An object photographing means for photographing the state of the object;
Comparison means for comparing the image photographed by the object photographing means with the reference image photographed by the reference state photographing means to generate comparison information;
A portable information processing program that functions as control means for controlling the information processing based on comparison information generated by the comparison means.

Images