JP6315122B2 - Display control apparatus, display control method, and program - Google Patents

Description

The present invention relates to a display control device, a display control method, and a program.

2. Description of the Related Art In recent years, a technique for deforming an object (target object) displayed on a display device based on pressure information applied by pressure input means has been developed.
For example, in Patent Document 1, a pressure-sensitive sensor sheet is attached to the surface of a basic structure, the pressure applied to the surface by a user's hand or the like is detected, and the object is processed into a graphic representation of the basic structure. A three-dimensional object shape generation method for applying deformation is disclosed. Patent Document 2 discloses a method for deforming an object displayed on a display device in accordance with an operation pressure input to a controller having a pressure sensing means.

Japanese Patent Laid-Open No. 6-232122 Special table 2003-519558 gazette

  However, in the conventional technique described above, the degree of deformation of the target object in the image with respect to the applied pressure is uniform regardless of the material characteristics, and thus the material characteristics of the target object cannot be experienced. It was.

An object of the present invention, by any operation against the object of interest is to be able to see whether the object is how to change.

The display control apparatus according to the present invention includes a display control unit that controls display of the first object image and the second object image, a detection unit that detects an arbitrary operation by the user for each object image, and the first control unit. A first simulation image based on a simulation result when the operation is performed on the first object corresponding to the object image, and a simulation result when the operation is performed on the second object corresponding to the second object image And a second simulation image obtained by the acquisition unit , wherein the detection unit detects an arbitrary operation by the user with respect to an arbitrary position in the first object image, and the acquisition unit detects the detected A first simulation image when operated with respect to the position of the first object corresponding to the position in the first object image and the position in the first object image Wherein the second simulated images in the case of performing the operation on the position in the second object image, and acquires the display control means, the second simulation obtained with the first simulation image by the acquisition unit It is characterized by controlling display of images.
Further, the display control method according to the present invention includes a display step for controlling display of the first object image and the second object image, a detection step for detecting an arbitrary operation by the user for each object image, A first simulation image based on a simulation result when the operation is performed on the first object corresponding to one object image, and a simulation when the operation is performed on the second object corresponding to the second object image A second simulation image obtained as a result, and an acquisition step for acquiring an arbitrary operation by a user for an arbitrary position in the first object image, wherein the acquisition step is detected. A first simulation image when operated on the position of the first object corresponding to the position in the first object image; and the first object A second simulation image obtained when the operation is performed on a position in the second object image corresponding to a position in the image, and the display step includes the first simulation acquired in the acquisition step. Display control of the image and the second simulation image is performed.
A program according to the present invention is a program for controlling a computer of a display control device, wherein the computer controls display control of a first object image and a second object image, and each object image. Detecting means for detecting an arbitrary operation by a user for the first object, a first simulation image based on a simulation result when the operation is performed on the first object corresponding to the first object image, and the second object image And a second simulation image based on a simulation result when the operation is performed on the corresponding second object. The detection unit is a user for an arbitrary position in the first object image. The acquisition means detects the position of the first object corresponding to the detected position in the first object image. A first simulation image in the case of being operated on, and a second simulation image in the case of performing the operation on a position in the second object image corresponding to a position in the first object image, The display control means acquires the first simulation image and the second simulation image acquired by the acquisition means, and controls display.

According to the present invention, by any operation against the object of interest, it is possible to check whether the object is how to change.

1 is a diagram illustrating an example of the overall configuration of an information processing system according to a first embodiment of the present invention. It is a block diagram which shows the function structure of the server apparatus which concerns on 1st Embodiment of this invention. It is a top view of the client terminal which concerns on 1st Embodiment of this invention. It is a schematic sectional drawing of the IV-IV line | wire of FIG. It is a block diagram which shows the function structure of the client terminal which concerns on 1st Embodiment. It is a figure which shows the structure of a material characteristic table. It is a flowchart of the image deformation process performed in the information processing system which concerns on 1st Embodiment. It is a figure which shows the structural example of an image deformation process condition input screen. It is a figure which shows a simulation screen. It is a schematic sectional drawing of the client terminal which concerns on 2nd Embodiment. It is a block diagram which shows the function structure of the client terminal which concerns on 2nd Embodiment. It is a flowchart of the image deformation process performed in the information processing system which concerns on 2nd Embodiment. It is a figure which shows the structural example of an image deformation process condition input screen. It is a figure which shows the simulation screen which concerns on twisting operation | movement. It is a figure which shows the simulation screen which concerns on knob operation. It is a block diagram which shows the function structure of the information processing apparatus which concerns on 3rd Embodiment. It is a flowchart of the image deformation process performed in the information processing apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the function structure of the information processing apparatus which concerns on 4th Embodiment. It is a flowchart of the image deformation process performed in the information processing apparatus which concerns on 4th Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First embodiment]
As illustrated in FIG. 1, the information processing system 1 includes a server device 100 and a client terminal 200. These devices are connected via a communication network N. The communication network N is a communication network N constructed using a dedicated line or an existing general public line, and various line forms such as a LAN (Local Area Network) and a WAN (Wide Area Network) can be applied. Is possible. The communication network N includes, for example, various communication network networks such as a telephone line network, ISDN line network, dedicated line, mobile communication, communication satellite line, CATV line network, IP network, VoIP (Voice over Internet Protocol). Includes gateways, internet service providers, etc.

  First, the functional configuration of the server apparatus 100 will be described with reference to FIG. As shown in FIG. 2, the server device 100 provides a CPU (Central Processing Unit) 110 that performs overall control of the device, an input unit 120 having a plurality of operation keys, and a working memory space for the CPU 110. A random access memory (RAM) 130, a display unit 140, a storage unit 150 that stores a control program and control data executed by the CPU 110, and a communication unit 160 are provided. CPU 110, input unit 120, RAM 130, display unit 140, storage unit 150, and communication unit 160 are connected via a bus 170.

  The CPU 110 reads the system program stored in the storage unit 150, expands the read program in the work area of the RAM 130, and controls each unit according to the system program. In addition, the CPU 110 reads out various processing programs stored in the program storage unit 151 of the storage unit 150, develops them in a work area, and executes processing. For example, the CPU 110 functions as an image control unit by executing an image deformation process described later.

  The input unit 120 includes a keyboard having cursor keys, characters, numeric input keys, various function keys, and the like. The input unit 120 accepts pressing input of each key by the operator and outputs the operation information to the CPU 110. The input unit 120 includes a pointing device such as a mouse, receives a position input, and outputs it to the CPU 110 as operation information.

  The RAM 130 is a volatile memory. The RAM 130 has a work area for storing various programs to be executed and data related to these various programs.

  The display unit 140 is configured by an LCD (Liquid Crystal Display) or the like, and performs various displays according to display information input from the CPU 110.

  The storage unit 150 is configured by an HDD (Hard Disk Drive) having a magnetic recording medium, a nonvolatile semiconductor memory, or the like. As shown in FIG. 2, the storage unit 150 is provided with a program storage unit 151, an image storage unit 153, a material property storage unit 154, and the like.

The program storage unit 151 stores various programs such as a system program executed by the CPU 110 and the server processing program 152, data necessary for executing these programs, and the like.
These programs are stored in the program storage unit 151 in the form of computer-readable program codes. The CPU 110 sequentially executes operations according to the program code.

  The server processing program 152 is a program for simulating the deformation of the image of the target object based on the pressure applied to the image of the target object (hereinafter referred to as an object).

  The image storage unit 153 stores image data of an object and image data of a reference object (hereinafter referred to as a hardness reference object).

  The hardness reference object is an object serving as a reference for material characteristics, in other words, a reference for comparison of hardness.

In the material property storage unit 154, a material property table T1 corresponding to the image data stored in the image storage unit 153 is stored.
Here, the material property storage unit 154 functions as a storage unit that stores the material property of the target object.

  As shown in FIG. 6, the material property table T1 has fields of an image number T11, an image name T12, a material T13, and hardness data T14. The material property table T1 stores material property data of the object and the hardness reference object.

  The image number T11 is identification information for identifying image data. The image name T12 is the name of the image with the image number T11. The material T13 is the material of the object related to the image with the image number T11. The hardness data T14 is the hardness data of the object related to the image having the image number T11, and is Vickers hardness, Young's modulus, rigidity, Poisson's ratio, and the like.

  The communication unit 160 serving as a server-side communication unit includes a modem, a router, a network card, and the like, is connected to the communication network N, and communicates with the client terminal 200 on the communication network N.

Next, the functional configuration of the client terminal 200 will be described with reference to FIGS.
The client terminal 200 includes a central processing unit (CPU) 210 that performs overall control of the device, a touch panel 220 that includes an input unit 221 and a display unit 225, and a pressure sensor unit 230 that detects pressure applied to the touch panel 220. A keyboard 260 having a plurality of operation keys, a RAM (Random Access Memory) 270 that provides a working memory space for the CPU 210, a storage unit 280 that stores a control program executed by the CPU 210, control data, and the like, Unit 295. The CPU 210, the touch panel 220, the pressure sensor unit 230, the keyboard 260, the RAM 270, the storage unit 280, and the communication unit 295 are connected via a bus 296.

The CPU 210 reads the system program stored in the storage unit 280, develops it in the work area of the RAM 270, and controls each unit according to the system program. Further, the CPU 210 reads out various processing programs stored in the program storage unit 281 of the storage unit 280, develops them in the work area, and executes the processing.
For example, the CPU 210 functions as a display control unit by executing processing on the client terminal 200 side in image deformation processing described later.

  The touch panel 220 includes an input unit 221 and a display unit 225. The display unit 225 includes an LCD (Liquid Crystal Display), an ELD (Electro Luminescent Display), and the like, and displays a display image such as a character input image in accordance with an instruction of a display signal input from the CPU 210. The input unit 221 receives a position input on the screen displayed on the display unit 225 by contact (pressing) of the user's finger or the like, and outputs the position (coordinate) information to the CPU 210. The input unit 221 is, for example, a resistance film method or a capacitance method.

  The pressure sensor unit 230 measures the applied pressure with a pressure sensitive element of a strain gauge resistance type, a semiconductor piezoresistive type, a capacitance type, a silicon resonant type, or the like, converts it into an electric signal, and outputs it. Here, the pressure sensor unit 230 functions as a pressure detection unit that detects the pressure applied to the touch panel 220.

  The keyboard 260 includes a cursor key, characters, numeric input keys, various function keys, and the like. The keyboard 260 accepts pressing input of each key by the user and outputs operation information to the CPU 210.

  The RAM 270 is a volatile memory. The RAM 270 also has a work area for storing various programs to be executed and data related to these various programs.

  The storage unit 280 includes an HDD (Hard Disk Drive) having a magnetic recording medium, a nonvolatile semiconductor memory, and the like. The storage unit 280 is provided with a program storage unit 281 as shown in FIG.

The program storage unit 281 stores various programs such as a system program executed by the CPU 110, the first terminal processing program 282, data necessary for executing these programs, and the like.
These programs are stored in the program storage unit 281 in the form of computer readable program codes. The CPU 210 sequentially executes operations according to the program code.

  The communication unit 295 serving as a terminal-side communication unit includes a modem, a router, a network card, and the like, is connected to the communication network N, and communicates with the client terminal 200 on the communication network N.

Next, the operation | movement of the image deformation process performed in the information processing system 1 of 1st Embodiment is demonstrated with reference to FIG.
The processing on the client terminal 200 side in the image transformation processing is executed in cooperation with the CPU 210 and the first terminal processing program 282 stored in the storage unit 280. Processing on the server device 100 side in the image transformation processing is executed in cooperation with the CPU 110 and the server processing program 152 stored in the storage unit 150.

  First, the CPU 210 of the client terminal 200 accesses the server device 100 via the communication unit 295 in response to an input instruction to the input unit 221, generates an image deformation processing condition input screen for the server device 100, and displays the screen. Is requested (step S1).

  When the CPU 110 of the server apparatus 100 receives a request for generating an image deformation processing condition input screen and a reply to the screen from the client terminal 200 from the communication unit 160, the CPU 110 generates an image deformation processing condition input screen. The data is transmitted to the requesting client terminal 200 (step S2).

  When the CPU 210 of the client terminal 200 receives the transmission from the server device 100 via the communication unit 295, the CPU 210 displays the image deformation processing condition input screen 2251 on the display unit 225 of the client terminal (step S3).

  As shown in FIG. 8, the image deformation processing condition input screen 2251 includes an image selection field 2251A, a pressure application shape selection field 2251B, a pressure application position selection field 2251C, a hardness reference object selection field 2251D, and an execution button. 2251E is displayed.

  The image selection column 2251A is a column for selecting an image of an object on which deformation processing is to be executed. The object image is selected from the pull-down menu in the image selection field 2251A. The pressure application shape selection column 2251B is a column for selecting an application shape when pressure is applied to an object image. The application shape displayed in the column is selected by a check button in the pressure application shape selection column 2251B. The default setting is the standard setting. The pressure application position selection column 2251C is a column for selecting to which position in the object image the pressure is to be applied by using a check button. The default setting is a standard setting, and is set at the center of the image, for example. The hardness reference object selection column 2251D is a column for selecting a hardness reference object by a check button. The default setting is a standard setting, and a predetermined hardness reference object is set. The execution button 2251E is a button for executing deformation processing under the selected condition.

  Here, the hardness reference object is held at the user's hand so that the hardness of the object can be simulated in a simulated manner via the hardness reference object. As the hardness reference object, a predetermined object is set as a standard, but from the viewpoint of comparison with the object image, a plurality of types are prepared and can be designated by the user. Specifically, the hardness reference object can be specified by the user from, for example, clay, sponge, etc. in consideration of easy procurement.

  Next, when the user selects a pressure application shape, an object image, or the like via the image transformation processing condition input screen 2251 and presses the execution button, the CPU 210 of the client terminal 200 presses the pressure application shape, the object image, or the like. Is selected (step S4). Here, the CPU 210, the display unit 225 that displays the pressure application shape selection field 2251 </ b> B, and the input unit 221 function as a selection unit that selects a pressure application shape indicating a shape in a range to which pressure is applied.

  Next, the CPU 210 of the client terminal 200 accesses the server device 100 via the communication unit 295, and requests the server device 100 to generate a simulation screen and return the screen (step S5).

  When the CPU 110 of the server device 100 receives a request for generating a simulation screen and returning the screen from the client terminal 200 from the communication unit 160, the CPU 110 generates a simulation screen and transmits the screen to the requesting client terminal 200 by the communication unit 160. (Step S6).

Next, the CPU 210 of the client terminal 200 displays a simulation screen (step S7).
On the simulation screen 2252, as shown in FIG. 9A, an image P of the selected object is displayed. Further, the hardness reference object image S is displayed in the hardness reference object display field 2252A.

  Next, the CPU 210 of the client terminal 200 determines whether or not contact with the touch panel 220 has been detected (step S8). When CPU 110 determines that contact with touch panel 220 is not detected (step 84; NO), it continues contact detection. On the other hand, as shown in FIG. 9B, when the finger of the user's hand H applies pressure on the object image P on the touch panel 220 with a finger or the like, the CPU 210 makes contact with the touch panel 220. When it detects (step S8; YES), the position data of the applied contact is acquired, and the position data is transmitted to the server apparatus 100 via the communication unit 295 (step S9). When the pressure application position selection is a standard setting, data to that effect is transmitted. Moreover, the pressure data is acquired and the server apparatus 100 transmits the data via the communication unit 295 (step S10).

  When the CPU 110 of the server device 100 receives the position data and pressure data from the client terminal 200 from the communication unit 160, the CPU 110 executes a simulation for the image of the object, and transmits the result to the requesting client terminal 200 through the communication unit 160. (Step S11). Further, the CPU 110 of the server apparatus 100 executes a simulation for the image S of the hardness reference object, and transmits the result to the requesting client terminal 200 via the communication unit 160 (step S12).

The simulation for the image P of the selected object is performed as follows. First, the CPU 110 of the server apparatus 100 refers to the material property table T1 to obtain hardness data related to the object image P and the reference object image S from the material property table T1 shown in FIG.
Next, a deformation simulation is performed on the object image P based on the pressure data, the position data, and the pressure application shape. As material characteristics, material T13 and / or hardness data T14 (for example, elastic modulus such as Young's modulus) can be used.

  Here, a simulation program included in the server processing program 152 is used for the simulation.

  Next, the CPU 210 of the client terminal 200 displays the image PA of the deformed object based on the simulation result (step S13).

  Next, the CPU 210 of the client terminal 200 displays an image SA of the deformed hardness reference object based on the simulation result (step S14).

  Specifically, as shown in FIG. 9C, the CPU 210 of the client terminal 200 displays modified images PA and SA related to the simulation result for both the object image and the hardness reference object image. indicate.

  Next, when the CPU 210 of the client terminal 200 does not detect contact on the touch panel 220 with respect to the image of the object displayed on the screen in step S15 (step S15; NO), the process ends. On the other hand, when a contact is detected in step S15 (step S15; YES), the process proceeds to step S9, and the processing after step S9 is performed to capture position data and pressure data and communicate with the server apparatus 100. Continue the simulation. In this way, when the user first applies a small pressure on the touch panel 220 with a finger or the like corresponding to the image P of the object, the CPU 110 of the server device 100 performs a deformation simulation and deforms the display unit 225 of the client terminal 200. When the screen is displayed and a larger pressure is applied, the deformation is further advanced.

As described above, according to the information processing system 1 according to the first embodiment, the information processing system includes the client terminal 200 and the server device 100 connected to the client terminal 200 through the communication network N so as to be able to transmit and receive data. In the system 1, the client terminal 200 includes a touch panel 220 having an input unit 221 having a touch input function, a display unit 225 disposed on the lower surface of the input unit 221, and a pressure for detecting a pressure applied to the touch panel 220. The CPU 210 of the client terminal includes a pressure sensor unit 230 as detection means, and displays an image of the target object on the display unit 225. The client terminal 200 is added to the touch panel 220 with information on the selected target object. Terminal that transmits information on pressure to server device 100 The CPU 110 of the server apparatus 100 includes a communication unit 295 as a communication unit, and the target object displayed on the display unit 225 based on the pressure detected by the pressure sensor unit 230 as a pressure detection unit as an image control unit. The server apparatus 100 generates a communication unit 160 serving as server-side communication means for transmitting the generated image of the target object after deformation to the client terminal 200 according to the material characteristics. The CPU 210 of the client terminal 200 displays the image of the deformed target object transmitted from the server apparatus 100 on the display unit 225 as display control means.
As a result, when pressure is applied to the target object related to the image, the deformation corresponding to the material characteristics of the target object can be expressed, so the user recognizes the applied pressure and the deformation of the target object in association with each other. This makes it possible to experience the material properties of the target object.

Further, according to the information processing system 1, the CPU 210 as the display control unit displays an image of the reference object indicating the material property reference on the display unit 225, and the CPU 110 as the image control unit is based on the detected pressure. Then, the image of the reference object after deformation according to the material characteristics of the reference object is generated, and the CPU 210 as the display control means displays the image of the reference object after deformation generated by the CPU 110 as the image control means. 225.
As a result, the image of the target object is deformed and the image of the reference object is deformed based on the pressure applied to the image of the target object, so that the hardness of the target object is simulated via the reference object. You can experience it. That is, it is possible to grasp the hardness of the target object by touching and deforming the actual object related to the reference object and comparing it with the simulation result.

Further, according to the information processing system 1, the server apparatus 100 includes the material property storage unit 154 as a storage unit that stores the material property of the target object, and the CPU 110 as the image control unit uses the material property of the target object as the material. Based on the pressure acquired from the characteristic storage unit 154 and detected by the pressure sensor unit 230, the deformation of the target object is simulated according to the material characteristics of the acquired target object, and the target object after deformation is based on the simulation result Generate an image of
Therefore, the deformation of the target object corresponding to the pressure can be continuously expressed, and the material characteristics of the target object can be more reliably realized.

Further, according to the information processing system 1, the input unit 221 of the touch panel 220 detects the position on the image of the target object to which pressure is applied, and the CPU 110 serving as the image control unit detects the target detected by the input unit 221. Based on the position of the object on the image, an image of the deformed target object is generated.
Thereby, the deformation position in the image of the target object can be arbitrarily set, and the material characteristics of the target object can be more reliably realized.

Further, according to the information processing system 1, when applying pressure to the image of the target object, the CPU 210 as a selection unit that selects the pressure application shape indicating the shape of the range to which pressure is applied, the pressure application shape selection field 2251B. The CPU 110 as an image control unit generates an image of the deformed target object based on the selected pressure application shape.
Thereby, the shape at the time of applying a pressure can be changed, and the material characteristic of a target object can be more reliably realized.

[Second Embodiment]
Next, an information processing system 2 according to the second embodiment of the present invention will be described.
The information processing system 2 according to the second embodiment has substantially the same configuration as the information processing system 1 according to the first embodiment, except for the details described below. Accordingly, the corresponding components are denoted by the corresponding reference numerals, and detailed description thereof is omitted.

  Although not shown, the information processing system 2 according to the second embodiment includes the server 100 of FIG. 1 and a client terminal 300, which are connected via a communication network. Although the plan view of the client terminal 300 is omitted, the schematic sectional view of FIG. 10 is a sectional view of the same place as FIG. As shown in FIG. 10, the client terminal 300 has a touch panel 220 (input unit 221 and A display unit 225) and a pressure sensor unit 230 are provided. In addition, a second input portion 240 and a second pressure sensor portion 250 are provided on the second surface portion 238 of the housing 290, that is, on the back surface side of the apparatus.

  In addition to the configuration of the client terminal 200 of the information processing system 1 according to the first embodiment, the client terminal 300 of the information processing system 2 according to the second embodiment, as shown in FIG. And a second pressure sensor unit 250 that detects the pressure applied to the second input unit 240. The program storage unit 281 stores a second terminal processing program 283. Here, the second pressure sensor unit 250 functions as a second pressure detection unit that detects the pressure applied to the second input unit 240.

Next, the operation | movement of the image deformation process performed in the information processing system 2 of 2nd Embodiment is demonstrated with reference to FIG.
Processing on the client terminal 200 side in the image transformation processing in the second embodiment is executed in cooperation with the CPU 210 and the second terminal processing program 283 stored in the storage unit 280. Processing on the server device 100 side in the image transformation processing is executed in cooperation with the CPU 110 and the server processing program 152 stored in the storage unit 150.

  Steps S21 to S36 in the flowchart of the image deformation process executed in the information processing system 2 correspond to steps S1 to S15 in the flowchart of the image deformation process executed in the information processing system 1. In the following, different processes will be mainly described.

First, the image deformation processing condition input screen displayed in step 23 is different from that in the first embodiment.
On the image deformation processing condition input screen 2253, as shown in FIG. 13, an image selection field 2253A, a hardness reference object selection field 2253B, and an execution button 2253C are displayed.

  In step S <b> 27, the CPU 210 of the client terminal 300 displays a simulation screen 2254 on the display unit 225 as shown in FIGS. 14A and 15A.

Next, in step S28, the CPU 210 of the client terminal 300 determines whether or not contact with the front and back input units has been detected. The input parts on the front and back sides are the input part 221 provided on the first surface part of the housing 290 and the second input part 240 provided on the second surface part opposite to the first surface part.
If the CPU 210 of the client terminal 300 determines that contact with the front and back input units is not detected (step S28; NO), the contact detection is continued. On the other hand, as shown in FIGS. 14B and 15B, pressure is applied to the part of the touch panel 220 on which the image P of the object is displayed and the corresponding part of the second input unit 240 with a finger or the like. When the CPU 210 of the client terminal 300 determines that a contact has been detected (step S28; YES), it acquires the position data of the contact in the input part on the front and back sides, and in either one of the input parts on the front or back side. When a tracing operation (drag operation) is detected, the operation amount of the tracing operation (drag operation) on the front and back input units is acquired, and the position data and the like are transmitted to the server device 100 via the communication unit 295. Transmit (step S29). Moreover, the pressure data in the input part of the front and back is acquired, and the pressure data is transmitted to the server apparatus 100 via the communication part 295 (step S30).

  Next, in step S31, the CPU 210 determines an input operation performed by the user. Specifically, when the operation amount of the tracing operation (drag operation) is transmitted in step S29, the CPU 210 determines that the operation is a twisting operation. The twisting operation is also called a twisting operation. On the other hand, when the operation amount of the tracing operation (drag operation) is not transmitted in step S29, the CPU 210 determines that the operation is a knob operation.

FIG. 14 shows a case where the user performs a twisting operation on the simulation screen 2254. FIG. 15 shows a case where the user performs a knob operation on the simulation screen 2254.
14B and 15B show a state in which pressure is applied to the image P of the object on the touch panel 220 with two fingers of the user's hand H via the front and back input units. ing. In the example of FIG. 15B, an arrow is displayed in the vicinity of the image P of the object, and it is considered that pressure is applied to the object image P in the direction of the arrow when contact with the front and back input portions is detected. .

  Next, as shown in FIGS. 14C and 15C, the CPU 210 of the client terminal 300 determines the deformed object image PA and the deformed hardness reference based on the simulation result received from the server device 100. The image SA of the object is displayed (Step S34, Step S35).

  Next, when the CPU 210 of the client terminal 300 does not detect contact with the input parts on the front and back with respect to the image of the object displayed on the screen in step S36 (step S36; NO), the process ends. On the other hand, when a contact is detected in step S36 (step S36; YES), the process proceeds to step S29, and the processing after step S29 is performed to capture position data and pressure data and communicate with the server apparatus 100. While continuing the simulation.

As described above, according to the information processing system 2 according to the second embodiment, the information processing system includes the client terminal 300 and the server device 100 connected to the client terminal 300 through the communication network N so as to be able to transmit and receive data. In the system 2, in addition to the configuration of the information processing system 1 according to the first embodiment, in the housing 290 in which the touch panel 220 is disposed on the first surface portion 228, the second surface portion 238 opposite to the first surface portion 228. A second input unit 240 having a touch input function, and a second pressure sensor unit 250 that is a second pressure detecting means for detecting a pressure applied to the second input unit 240,
The CPU 110 as the image control unit of the server apparatus 100 is configured to change the target after deformation based on the pressure detected by the pressure sensor unit 230 as the pressure detection unit and the second pressure sensor unit 250 as the second pressure detection unit. Generate an image of the object.
As a result, it is possible to input a knob operation, and when the input is performed, it is possible to experience deformation according to the material characteristics of the target object. In addition to detecting pressure data, position data and manipulated variable can be input to input twisting motion (twisting motion). When this input is made, the user can experience deformation according to the material characteristics of the target object. be able to.

[Third Embodiment]
Next, an information processing apparatus 400 according to the third embodiment of the present invention will be described with reference to FIG.

  As illustrated in FIG. 16, the information processing apparatus 400 according to the third embodiment is configured by integrating the functions of the server apparatus 100 and the client terminal 200 according to the first embodiment. An information processing apparatus 400 according to the third embodiment has substantially the same configuration as the client terminal 200 in the first embodiment. Therefore, FIG. 3 and FIG. 4 are used for a plan view and a schematic cross-sectional view of the information processing device 400. In FIG. 16, components corresponding to those in FIG.

The CPU 210 reads the system program stored in the storage unit 280, expands the read system program in the work area of the RAM 270, and controls each unit according to the system program. Further, the CPU 210 reads out various processing programs stored in the program storage unit 281 of the storage unit 280, develops them in the work area, and executes the processing.
For example, the CPU 210 functions as a display control unit and an image control unit by executing an image deformation process described later.

An image storage unit 286 and a material property storage unit 287 are stored in the storage unit 280 of the information processing apparatus 400. The program storage unit 281 stores a third processing program 284.
Here, the material property storage unit 287 functions as a storage unit that stores the material property of the target object.

  Next, the operation | movement of the image deformation process performed in the information processing apparatus 400 of 3rd Embodiment is demonstrated with reference to FIG. Such image deformation processing is executed in cooperation with the CPU 210 and the third processing program 284 stored in the storage unit 280.

First, when image deformation processing is started, the CPU 210 displays an image deformation processing condition input screen 2251 on the display unit 225 (step S41).
The image deformation processing condition input screen displayed on the display unit 225 of the information processing apparatus 400 is the same as the image deformation processing condition input screen 2251 of FIG. 8 in the first embodiment.

  Next, when the user selects a pressure application shape, an object image, or the like via the image deformation processing condition input screen 2251 and presses the execution button, the CPU 210 selects the pressure application shape, the object image, or the like ( Step S42). Here, the CPU 210, the display unit 225 that displays the pressure application shape selection field 2251 </ b> B, and the input unit 221 function as a selection unit that selects a pressure application shape indicating a shape in a range to which pressure is applied.

Next, the CPU 210 displays a simulation screen (step S43).
The simulation screen displayed on the display unit 225 of the information processing apparatus 400 in step S43 is the same as the simulation screen 2252 shown in FIG.

  Next, the CPU 210 determines whether or not contact with the touch panel 220 has been detected (step S44). When CPU 210 determines that contact with touch panel 220 is not detected (step S44; NO), it continues contact detection. On the other hand, when the CPU 210 detects contact with the touch panel 220 by applying pressure with a finger or the like to the portion on the touch panel 220 on which the object image P is displayed (step S44; YES), the applied contact is performed. Position data is acquired (step S45), and the pressure data is acquired (step S46).

  Next, the CPU 210 performs a simulation for the image P of the selected object (step S47).

  Next, the CPU 210 performs a simulation for the image S of the hardness reference object under the same conditions as those for the image P of the object (step S48).

  Next, the CPU 210 displays the image PA of the deformed object based on the simulation result (step S49).

  Next, the CPU 210 displays an image SA of the deformed hardness reference object based on the simulation result (step S50).

  If the CPU 210 determines in step S51 that contact with the touch panel 220 is not detected with respect to the image of the object displayed on the screen (step S51; NO), the process ends. On the other hand, if it is determined in step S51 that contact has been detected (step S51; YES), the process proceeds to step S45, and the process is continued from step S45, thereby continuing the simulation while taking in position data and pressure data.

As described above, the information processing apparatus 400 according to the third embodiment includes the touch panel 220 including the input unit 221 having the touch input function and the display unit 225 disposed on the lower surface of the input unit 221, and the touch panel 220. A pressure sensor unit 230 as a pressure detection unit that detects the applied pressure, and the CPU 210 as a display control unit displays an image of the target object on the display unit 225, and the CPU 210 as the image control unit is detected. Based on the measured pressure, the image of the target object after deformation corresponding to the material characteristics of the target object displayed on the display unit 225 is acquired, and the CPU 210 as the display control unit acquires the acquired target object after deformation. The image is displayed on the display unit 225.
As a result, when pressure is applied to the target object related to the image, the deformation corresponding to the material characteristics of the target object can be expressed, so the user recognizes the applied pressure and the deformation of the target object in association with each other. This makes it possible to experience the material properties of the target object.

Further, the CPU 210 as the display control means displays an image of the reference object indicating the reference of the material property on the display unit 225, and the CPU 210 as the image control means changes the material property of the reference object based on the detected pressure. The CPU 210 as a display control unit acquires an image of the reference object after the deformation, and displays the acquired image of the reference object after the deformation on the display unit 225.
As a result, the image of the target object is deformed and the image of the reference object is deformed based on the pressure applied to the image of the target object, so that the hardness of the target object is simulated via the reference object. You can experience it. That is, it is possible to grasp the hardness of the target object by touching and deforming the actual object related to the reference object and comparing it with the simulation result.

The information processing apparatus 400 includes a material property storage unit 287 as a storage unit that stores the material property of the target object, and the CPU 210 as an image processing unit acquires the material property of the target object from the material property storage unit 287. Based on the detected pressure, the deformation of the target object is simulated according to the material characteristics of the acquired target object, and an image of the deformed target object is generated and acquired based on the simulation result.
Therefore, the deformation of the target object corresponding to the pressure can be continuously expressed, and the material characteristics of the target object can be more reliably realized.

Further, the input unit 221 of the touch panel 220 detects a position on the image of the target object to which pressure is applied, and the CPU 210 as an image processing unit is based on the position of the target object on the image detected by the input unit 221. Then, an image of the deformed target object is generated and acquired.
Thereby, the deformation position in the image of the target object can be arbitrarily set, and the material characteristics of the target object can be more reliably realized.

Further, when applying pressure to the image of the target object, the information processing apparatus 400 displays a CPU 210 and a pressure application shape selection column 2251B as selection means for selecting a pressure application shape indicating the shape of the range to which pressure is applied. The CPU 210 as an image processing unit generates and acquires an image of the target object after deformation based on the selected pressure application shape.
Thereby, the shape at the time of applying a pressure can be changed, and the material characteristic of a target object can be more reliably realized.

[Fourth Embodiment]
Next, an information processing apparatus 500 according to the fourth embodiment of the present invention will be described with reference to FIG.

  As illustrated in FIG. 18, the information processing apparatus 500 according to the fourth embodiment is configured by integrating the function of the server apparatus 100 and the function of the client terminal 300 in the second embodiment. An information processing apparatus 500 according to the fourth embodiment has substantially the same configuration as the client terminal 200 of the second embodiment. Therefore, FIG. 3 and FIG. 10 are used for a plan view and a schematic cross-sectional view of the information processing apparatus 500. In FIG. 18, components corresponding to those in FIG.

The CPU 210 reads the system program stored in the storage unit 280, expands the read system program in the work area of the RAM 270, and controls each unit according to the system program. Further, the CPU 210 reads out various processing programs stored in the program storage unit 281 of the storage unit 280, develops them in the work area, and executes the processing.
For example, the CPU 210 functions as a display control unit and an image control unit by executing an image deformation process described later.

An image storage unit 286 and a material property storage unit 287 are stored in the storage unit 280 of the information processing apparatus 500. The program storage unit 281 stores a fourth processing program 285.
Here, the material property storage unit 287 functions as a storage unit that stores the material property of the target object.

  Next, the operation | movement of the image deformation process performed in the information processing apparatus 500 of 4th Embodiment is demonstrated with reference to FIG. Such image deformation processing is executed in cooperation with the CPU 210 and the fourth processing program 285 stored in the storage unit 280.

  Steps S61 to S72 in the flowchart of the image deformation process executed in the information processing apparatus 500 are processes corresponding to steps S41 to S51 in the flowchart of the image deformation process executed in the information processing apparatus 400. In the following, different processes will be mainly described.

  First, when image deformation processing is started, the CPU 210 displays an image deformation processing condition input screen on the display unit 225 (step S61). The image deformation processing condition input screen displayed on the display unit 225 of the information processing apparatus 500 is the same as the image deformation processing condition input screen 2253 of FIG. 13 in the second embodiment.

  In addition, the simulation screen displayed on the display unit 225 of the information processing apparatus 500 in step S63 is the same as the simulation screen 2254 shown in FIGS.

In step S64, CPU 210 determines whether or not contact with the front and back input units has been detected. The input parts on the front and back sides are the input part 221 provided on the first surface part of the housing 290 and the second input part 240 provided on the second surface part opposite to the first surface part.
When CPU 210 determines that contact with the input parts on the front and back sides is not detected (step S64; NO), it continues the contact detection. On the other hand, when pressure is applied with a finger or the like to the part of the touch panel 220 on which the image P of the object is displayed and the corresponding part of the second input unit 240, the CPU 210 determines that contact has been detected (step S64; YES), the contact position data on the input part on the front and back sides is acquired, and if a tracing operation (drag operation) is detected on one of the front and back input parts, The operation amount of the tracing operation (drag operation) is acquired (step S65). Further, pressure data at the front and back input sections is acquired (step S66).

  Next, in step S67, the CPU 210 determines an input operation performed by the user. Specifically, when the operation amount of the tracing operation (drag operation) is transmitted in step S65, the CPU 210 determines that the operation is a twisting operation. The twisting operation is also called a twisting operation. On the other hand, when the operation amount of the tracing operation (drag operation) is not transmitted in step S65, the CPU 210 determines that the operation is a knob operation.

  In steps S70 and S71, the CPU 210 displays a deformed object image PA and a deformed hardness reference object image SA.

  If the CPU 210 determines in step S72 that contact with the front and back input units is not detected with respect to the image of the object displayed on the screen (step S72; NO), the process ends. On the other hand, if it is determined in step S72 that contact has been detected (step S72; YES), the process proceeds to step S65, and the processing after step S65 is performed to capture position data and pressure data and continue the simulation. .

As described above, according to the information processing apparatus 500 according to the fourth embodiment, in addition to the configuration of the information processing apparatus 400 according to the third embodiment, in the casing 290 in which the touch panel 220 is disposed on the first surface portion 228, A second input unit 240 having a touch input function and a second pressure detection means for detecting the pressure applied to the second input unit 240, disposed on the second surface unit 238 opposite to the first surface unit 228. The CPU 210 as the image control means includes the second pressure sensor part 250 and the pressure detected by the pressure sensor part 230 as the pressure detection means and the second pressure sensor part 250 as the second pressure detection means. Based on this, an image of the deformed target object is generated.
As a result, it is possible to input a knob operation, and when the input is performed, it is possible to experience deformation according to the material characteristics of the target object. In addition to detecting pressure data, position data and manipulated variable can be input to input twisting motion (twisting motion). When this input is made, the user can experience deformation according to the material characteristics of the target object. be able to.

The present invention is not limited to the above-described embodiment, and various modifications can be made. For example, in the present embodiment, as an example in which the image control unit generates a deformed image, a configuration is shown in which a simulation is performed each time based on a predetermined pressure application condition to generate a deformed target object image. . On the other hand, a plurality of image data deformed stepwise corresponding to a plurality of pressure application conditions is prepared in advance, and an image close to the input pressure application condition is adopted to obtain a deformed image. Can be configured. Specifically, a deformed object image P and a hardness reference object image S are prepared in advance for a plurality of conditions in which the pressure application shape, the pressure application position, the pressure, and the like are changed. Then, an image having a condition close to the acquired pressure data or the like can be selected to display the image P of the object and the deformed image of the hardness reference object. Such image generation is also included in image generation in claims 1 and 7 of the present application.
Thereby, based on the stress application conditions, it is possible to perform processing more easily than in the case where simulation is performed each time, and the processing time can be shortened.
In addition, the selection of the hardness reference object is configured to be arbitrarily specified by the user, but the hardness reference object closer to the material property of the target object is automatically selected from a plurality of hardness reference objects. May be. By comprising in this way, the material characteristic of a target object can be felt more reliably.

Although the embodiment of the present invention has been described, the scope of the present invention is not limited to the above-described embodiment, and is included in the invention described in the claims and the equivalent scope thereof. Hereinafter, the invention described in the scope of claims of the present application will be appended.
[Appendix]
<Claim 1>
A touch panel having an input unit having a touch input function and a display unit disposed on a lower surface of the input unit;
Display control means for displaying an image of a target object on the display unit;
Pressure detecting means for detecting pressure applied to the touch panel;
Image control means for generating an image of the target object after deformation according to the material characteristics of the target object displayed on the display unit, based on the pressure detected by the pressure detection means,
The information processing apparatus, wherein the display control unit displays an image of the target object after deformation generated by the image control unit on the display unit.
<Claim 2>
The display control means displays an image of a reference object indicating a material property reference on the display unit,
The image control means generates an image of the reference object after deformation according to the material characteristics of the reference object based on the pressure detected by the pressure detection means,
The information processing apparatus according to claim 1, wherein the display control unit displays an image of the reference object after the deformation generated by the image control unit on the display unit.
<Claim 3>
Storage means for storing the material properties of the target object;
The image control means includes
The material property of the target object is acquired from the storage unit, and based on the pressure detected by the pressure detection unit, the deformation of the target object is simulated according to the acquired material property of the target object, and the simulation result The information processing apparatus according to claim 1, wherein an image of the target object after deformation is generated based on the information.
<Claim 4>
The input unit detects a position on the image of the target object to which pressure is applied,
The information processing apparatus according to claim 3, wherein the image control unit generates a deformed image of the target object based on a position on the image of the target object detected by the input unit.
<Claim 5>
When applying pressure to the image of the target object, comprising a selection means for selecting a pressure application shape indicating a shape of a range to which pressure is applied,
The information processing apparatus according to claim 3, wherein the image control unit generates an image of the target object after the deformation based on the pressure application shape selected by the selection unit.
<Claim 6>
A housing in which the touch panel is disposed on the first surface;
A second input part having a touch input function, and a second pressure detecting means for detecting a pressure applied to the second input part, disposed on a second surface part opposite to the first surface part of the housing. And having
The said image control means produces | generates the image after the deformation | transformation of the said target object based on each pressure detected by the said pressure detection means and the said 2nd pressure detection means, The Claim 1 or 2 characterized by the above-mentioned. Information processing device.
<Claim 7>
In an information processing system comprising a client terminal and a server device connected to the client terminal via a communication network so as to be able to transmit and receive data,
The client terminal is
A touch panel having an input unit having a touch input function and a display unit disposed on a lower surface of the input unit;
Pressure detecting means for detecting pressure applied to the touch panel;
Display control means for displaying an image of a target object on the display unit;
Terminal-side communication means for transmitting information on the target object and information on pressure applied to the touch panel to the server device,
The server device
Based on the pressure detected by the pressure detection means, an image control means for generating an image of the target object after deformation according to material characteristics of the target object displayed on the display unit;
Server-side communication means for transmitting an image of the target object after deformation generated by the image control means to a client terminal,
The information processing system, wherein the display control unit of the client terminal displays an image of the target object after deformation transmitted from the server device on the display unit.
<Claim 8>
The display control means displays an image of a reference object indicating a material property reference on the display unit,
The image control means generates an image of the reference object after deformation according to the material characteristics of the reference object based on the pressure detected by the pressure detection means,
The information processing system according to claim 7, wherein the display control unit displays the deformed image of the reference object generated by the image control unit on the display unit.
<Claim 9>
The server device
Storage means for storing the material properties of the target object;
The image control means includes
The material property of the target object is acquired from the storage unit, and based on the pressure detected by the pressure detection unit, the deformation of the target object is simulated according to the acquired material property of the target object, and the simulation result The information processing system according to claim 7 or 8, wherein an image of the target object after deformation is generated based on the information.
<Claim 10>
The input unit detects a position on the image of the target object to which pressure is applied,
The information processing system according to claim 9, wherein the image control unit generates an image of the target object after deformation based on a position on the image of the target object detected by the input unit.
<Claim 11>
The client terminal is
When applying pressure to the image of the target object, comprising a selection means for selecting a pressure application shape indicating a shape of a range to which pressure is applied,
The information processing system according to claim 9 or 10, wherein the image control unit generates an image of the target object after the deformation based on the pressure application shape selected by the selection unit.
<Claim 12>
The client terminal is
A housing in which the touch panel is disposed on the first surface;
A second input part having a touch input function, and a second pressure detecting means for detecting a pressure applied to the second input part, disposed on a second surface part opposite to the first surface part of the housing. And having
9. The image control unit according to claim 7, wherein the image control unit generates an image of the target object after the deformation based on the pressures detected by the pressure detection unit and the second pressure detection unit. The information processing system described.
<Claim 13>
A touch panel having an input unit having a touch input function and a display unit disposed on a lower surface of the input unit;
Pressure detecting means for detecting pressure applied to the touch panel;
A computer of an information processing apparatus equipped with
Display control means for displaying an image of a target object on the display unit;
Image control means for generating an image of the target object after deformation according to the material characteristics of the target object displayed on the display unit, based on the pressure detected by the pressure detection means;
A program for functioning as
The display control means is a program for displaying an image of the target object after deformation generated by the image control means on the display unit.

1, 2 Information processing system 100 Server device 110 CPU (image control means)
120 Input unit 130 RAM
140 Display Unit 150 Storage Unit 151 Program Storage Unit 152 Server Processing Program 153 Image Storage Unit 154 Material Characteristic Storage Unit (Storage Unit)
160 communication unit (server side communication means)
200, 300 Client terminal 210 CPU (display control means)
220 Touch Panel 221 Input Unit 225 Display Unit 230 Pressure Sensor Unit 240 Second Input Unit 250 Second Pressure Sensor Unit 260 Keyboard 270 RAM
280 Storage unit 281 Program storage unit 282 First terminal processing program 283 Second terminal processing program 290 Case 295 Communication unit (terminal side communication means)
296 Bus 400, 500 Information processing device 210 CPU (display control means, image control means)
284 Third processing program 285 Fourth processing program 286 Image storage unit 287 Material characteristic storage unit (storage means)

Claims (7)

Display control means for controlling display of the first object image and the second object image;
Detecting means for detecting an arbitrary operation by the user for each object image;
A first simulation image based on a simulation result when the operation is performed on the first object corresponding to the first object image, and a case where the operation is performed on the second object corresponding to the second object image An acquisition means for acquiring a second simulation image based on the simulation result of
Comprising
The detecting means detects an arbitrary operation by a user for an arbitrary position in the first object image,
The acquisition means corresponds to a first simulation image when operated with respect to the position of the first object corresponding to the detected position in the first object image, and a position in the first object image. Obtaining a second simulation image when the operation is performed on a position in the second object image;
The display control means controls display of the first simulation image and the second simulation image acquired by the acquisition means.
A display control device characterized by that. The first object image is an image of a first object arbitrarily selected by a user,
The second object image is an image of a second object identified for comparison with the first object;
The display control apparatus according to claim 1. The acquisition means acquires a simulation image as a result of simulating how each object changes based on the material properties of each object.
The display control apparatus according to claim 1, wherein the display control apparatus is a display control apparatus. The detection means detects a pressure by an arbitrary pressure operation by a user,
The acquisition unit acquires a simulation image as a result of simulating how each object is deformed according to the detected pressure,
The display control apparatus according to any one of claims 1 to 3 . The display control means controls display of the simulation images of the objects instead of the object images of the objects,
The display control apparatus according to claim 1, wherein the display control apparatus is a display control apparatus. A display step for controlling display of the first object image and the second object image;
A detection step of detecting an arbitrary operation by the user for each object image;
A first simulation image based on a simulation result when the operation is performed on the first object corresponding to the first object image, and a case where the operation is performed on the second object corresponding to the second object image An acquisition step of acquiring a second simulation image based on the simulation result of
Have
The detection step detects an arbitrary operation by a user for an arbitrary position in the first object image,
The acquisition step corresponds to a first simulation image when operated with respect to the position of the first object corresponding to the detected position in the first object image, and a position in the first object image. Obtaining a second simulation image when the operation is performed on a position in the second object image;
The display step controls display of the first simulation image and the second simulation image acquired in the acquisition step.
A display control method characterized by the above. A program for controlling a computer of a display control device,
The computer,
Display control means for controlling display of the first object image and the second object image;
Detecting means for detecting an arbitrary operation by the user for each object image;
A first simulation image based on a simulation result when the operation is performed on the first object corresponding to the first object image, and a case where the operation is performed on the second object corresponding to the second object image An acquisition means for acquiring a second simulation image based on the simulation result of
Function as
The detecting means detects an arbitrary operation by a user for an arbitrary position in the first object image,
The acquisition means corresponds to a first simulation image when operated with respect to the position of the first object corresponding to the detected position in the first object image, and a position in the first object image. Obtaining a second simulation image when the operation is performed on a position in the second object image;
The display control means controls display of the first simulation image and the second simulation image acquired by the acquisition means.
A program characterized by that.

Images