JP2012143447A - Network system, control method, controller, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a novel network system using a model device that moves based on a command from a controller.SOLUTION: The controller 100 receives a command for moving the model device and transmits the command to the model device. The model device 200 moves the model device based on the command from the controller, captures an image of an area in front of the model device, acquires the position of the model device, acquires the direction of the model device, and transmits the captured image, the position, and the direction to the controller. The controller creates a virtual image for display from data of a course of the model device based on the position and the direction received from the model device, and displays, on a display 130, an image formed by composing the captured image received from the model device and the virtual image.

Description

  The present invention relates to a technique for controlling the operation of a model apparatus by radio control.

  Techniques for controlling model devices such as sports cars, helicopters and robots by radio control are known.

  For example, Japanese Unexamined Patent Application Publication No. 2004-42148 (Patent Document 1) discloses a mobile robot. According to Japanese Patent Laying-Open No. 2004-42148 (Patent Document 1), a self-orientation detecting unit, a moving unit, and an obstacle detecting unit are mounted on a mobile robot. An environmental map storage means and a route planning means are incorporated in an external device of the mobile robot. Through the communication between the mobile robot and an external device, the route search from the self-location to the destination with reference to the environment map, the movement of the mobile robot along the planned route, the avoidance motion for unknown obstacles not on the environment map, the planned route A new route search from the new self-position after deviating from the route to the moving destination, and movement along the re-planned route are performed.

  JP-A-61-70617 (Patent Document 2) discloses a course guidance control system for an automatic traveling body. According to Japanese Patent Application Laid-Open No. 61-70617 (Patent Document 2), a speed control method using a traveling speed as a function of a steering angle is introduced, and a traveling speed control corresponding to the steering angle is performed. It can be used as a self-guidance control method for all traveling objects regardless of the use / type of traveling object to be controlled or conditions of the traveling surface, for example, floor or on-road or off-road.

JP 2004-42148 A JP-A-61-70617

  New network systems using model devices that move based on commands from controllers are being sought.

  The present invention has been made to solve such a problem, and an object thereof is to provide a new network system using a model device that moves based on a command from a controller.

  According to one aspect of the present invention, a network system including a model device and a controller is provided. The model device includes a moving unit for moving the model device based on a command from the controller, a camera for photographing the front of the model device, a GPS unit for acquiring the position of the model device, A compass for acquiring the orientation, and a first communication interface for receiving a command from the controller and transmitting a captured image, position, and orientation from the camera to the controller. The controller includes a display, an operation unit for receiving commands, a second communication interface for communicating with the model device, a memory for storing course data for the model device, and a position and orientation received from the model device. And a processor for generating a virtual image for display from the course data and displaying on the display an image obtained by synthesizing the captured image received from the model device and the virtual image.

  Preferably, the processor creates course data for the model device along the position trajectory based on the time-series data of the position received from the model device.

  Preferably, the memory stores course data for a plurality of model devices. The processor accepts a course selection command for a plurality of model devices via the operation unit.

  Preferably, the processor transmits the course data for the model device to the other controller via the second communication interface.

  Preferably, the processor receives course data for the model device from another controller via the second communication interface.

  Preferably, the processor receives, from the other controller via the second communication interface, time series data in which the position and orientation of the other model apparatus and the elapsed time from the start time are associated with each other, and the time series data To obtain the position and orientation of the other model device corresponding to the elapsed time from the start time of the model device, and based on the position and orientation received from the model device and the position and orientation of the other model device. An image showing the model device is added to the virtual image for display.

  When another situation of this invention is followed, the control method in a network system provided with the model apparatus and the controller which memorize | stores the data of the course for model apparatuses is provided. The control method includes a step in which the controller receives a command for moving the model device, a step in which the controller transmits a command to the model device, and a step in which the model device moves the model device based on the command from the controller. A step in which the model device photographs the front of the model device; a step in which the model device acquires the position of the model device; a step in which the model device acquires the orientation of the model device; Transmitting the position and orientation to the controller, creating a virtual image for display from the course data for the model device based on the position and orientation received from the model device by the controller, and the controller The composite image of the captured image received from the model device and the virtual image is displayed on the display. And a step.

  According to another aspect of the present invention, a display, a memory for storing course data for the model device, an operation unit for receiving a command for moving the model device, a command is transmitted to the model device, Based on the communication interface for receiving the photographed image and the position and orientation of the model device from the device, and the position and orientation received from the model device, a virtual image for display is created from the course data, and from the model device A controller is provided that includes a processor for causing a display to display an image obtained by combining a received captured image and a virtual image.

  According to another aspect of the present invention, there is provided a control method in a controller including a memory for storing course data for a model apparatus, a display, a communication interface, and a processor. The control method includes a step in which a processor receives a command for moving the model device, a step in which the processor transmits a command to the model device via a communication interface, and a processor in which the model device is transmitted via the communication interface. Receiving the captured image and the position and orientation of the model device from the processor, the processor creating a virtual image for display from the course data based on the position and orientation received from the model device, and the processor And displaying on the display an image obtained by synthesizing the captured image received from the model device and the virtual image.

  According to another aspect of the present invention, there is provided a control program for causing a controller including a memory for storing course data for a model device, a display, a communication interface, and a processor to control the model device. The control program includes a step of receiving a command for moving the model device to the processor, a step of transmitting a command to the model device via the communication interface, and a captured image and a model device from the model device via the communication interface. A step of receiving a position and orientation of the image, a step of creating a virtual image for display from the course data based on the position and orientation received from the model device, a captured image and a virtual image received from the model device, And a step of displaying the synthesized image on the display.

  As described above, according to the present invention, a new network system using a model device that moves based on a command from a controller is realized.

It is an image figure which shows the whole structure of the network system 1 which concerns on this Embodiment. It is a block diagram showing the hardware constitutions of the controller 100 which concerns on this Embodiment. It is an image figure which shows the state which the user which concerns on this Embodiment inputs the command for creating the course for the radio controlled car 200. FIG. It is an image figure which shows the controller 100 in the normal mode and race mode which concern on this Embodiment. It is an image figure which shows the controller 100 just before the start of the race which concerns on this Embodiment. It is an image figure which shows the controller 100 when the radio controlled car 200 concerning this Embodiment is located just before the goal point of a race. It is a block diagram showing the hardware constitutions of the radio controlled car 200 which concerns on this Embodiment. 4 is a sequence diagram showing a control method in controller 100 according to Embodiment 1. FIG. It is an image figure which shows the database 123 of the time series data for a race of the position and direction of another radio controlled car memorize | stored in the memory 120 which concerns on Embodiment 2. FIG. 6 is a sequence diagram illustrating a control method in the controller 100 according to Embodiment 2. FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

  In the following, a radio controlled car will be described as a representative example of the “model device”. However, the “model device” may be a helicopter or airplane that can fly, or a robot that can walk. In the case of a helicopter or an airplane, it is preferable to include an altimeter that acquires the altitude of the model device. In addition, the “controller” may be dedicated to control of the model apparatus, and has a display and communication interface such as a portable telephone, personal computer, electronic notebook, PDA (Personal Digital Assistant) having other functions. It may be a device.

[Embodiment 1]
<Overview of network system operation>
First, an outline of the operation of the network system 1 according to the present embodiment will be described. FIG. 1 is an image diagram showing an overall configuration of a network system 1 according to the present embodiment.

  Referring to FIG. 1, network system 1 includes controllers 100X and 100Y and radio controlled cars 200X and 200Y. Hereinafter, the controllers 100X and 100Y are collectively referred to as the controller 100 for the sake of explanation. The radio controlled cars 200X and 200Y are collectively referred to as a radio controlled car 200.

  Each user places the radio controlled car 200 on the ground of a park or a garden. The user controls the operation of the radio controlled car 200 via the controller 100.

  The radio controlled car 200 is equipped with a camera for photographing the front of the radio controlled car 200. The radio controlled car 200 transmits captured images to the controller 100 sequentially.

  The radio controlled car 200 is equipped with a GPS (Global Positioning System) for measuring the current position of the radio controlled car 200. The radio controlled car 200 sequentially transmits the current position to the controller 100.

  The radio controlled car 200 is equipped with an electronic compass for measuring the current direction (attitude) of the radio controlled car 200. The radio controlled car 200 transmits the current direction to the controller 100 sequentially. The radio controlled car 200 may transmit the captured image, the current position, and the current orientation to the controller 100 at the same time or separately.

  The controller 100 displays a captured image from the radio controlled car 200. The user inputs a command (a forward command, a reverse command, a direction change command, an acceleration / deceleration command, hereinafter these commands are also referred to as a movement command) for moving the radio controlled car 200 to the controller 100 while viewing the captured image. . The controller 100 transmits a movement command from the user to the radio controlled car 200.

  In the present embodiment, the controller 100 accumulates time series data (course creation time series data) of the current position from the radio controlled car 200. The controller 100 acquires the trajectory 301 of the radio controlled car 200 based on the time series data. The controller 100 creates data indicating a course (circuit) 302 for the radio controlled car 200 based on the trajectory 301 of the radio controlled car 200.

  Data indicating a course (also referred to as course data) includes a 3D object such as a white line indicating a course track or a course pylon. The course position, shape, and orientation are associated with actual map data. Alternatively, the course position, shape, and orientation are associated with latitude and longitude.

  However, the controller 100 may accept a course creation command via the touch panel 130 or the like. For example, the controller 100 receives a slide operation on the map from the user while the map is displayed. The controller 100 determines the position, shape, and orientation of the course based on the slide operation.

  Alternatively, the controller 100 may store a plurality of course data in advance. That is, even if the radio controlled car 200 does not run, the user may select a desired course from a plurality of courses prepared in advance. In this case, for example, the position, shape, and orientation of the course are associated with map data or latitude / longitude in advance. Alternatively, only the shape of the course is prepared, and when the user selects a course, the position and orientation of the course may be designated by the user.

  After the course is determined, the controller 100 starts a race based on a user instruction. The controller 100 acquires the position and orientation of the radio controlled car 200 in the course based on the course data and the current position and orientation of the radio controlled car 200. Based on the position and orientation of the radio controlled car 200 in the course, the controller 100 creates a virtual image indicating the 3D object viewed from the radio controlled car 200 from the course data.

  For example, after creating 3D model data of a circuit, the controller 100 determines the viewpoint of the 3D model data based on the position and orientation of the radio controlled car 200 on the circuit. The controller 100 creates a display image (virtual image) of 3D model data from the viewpoint.

  The controller 100 displays the virtual image superimposed on the captured image of the radio controlled car 200.

  In the present embodiment, the first controller 100X for controlling the first radio controlled car 200X can communicate with the second controller 100Y for controlling the second radio controlled car 200Y. The first controller 100X transmits course data to the second controller 100Y and receives course data from the second radio controlled car 200.

  That is, the first controller 100X and the second controller 100Y can control the traveling of the radio controlled cars 200X and 200Y based on the common course data. For example, the user of the first controller 100X can perform a race along the course while viewing the captured image from the first radio controlled car 200X, that is, while viewing the actual video of the second radio controlled car 200Y. it can. The same applies to the second controller 100Y.

  As described above, in the network system 1 according to the present embodiment, the user can control the movement of the radio controlled car 200 with the line of sight of the radio controlled car 200 while viewing the image (virtual image) indicating the virtual course. it can. As a result, the user can make the control of the movement of the radio controlled car 200 full of realism.

  Hereinafter, a specific configuration of the network system 1 for realizing such a function will be described in detail.

<Configuration of Controller 100>
One aspect of a specific configuration of the controller 100 will be described. FIG. 2 is a block diagram showing a hardware configuration of controller 100 according to the present embodiment. Referring to FIG. 2, controller 100 includes a CPU 110, a memory 120, a touch panel 130, a speaker 140, a button 150, a memory interface 160, a communication interface 170, and a clock 180 as main components. .

  The memory 120 is realized by various types of RAM (Random Access Memory), ROM (Read-Only Memory), a hard disk, and the like. The memory 120 stores a program executed by the CPU 110, map data, various model data for indicating a virtual course, and the like. In other words, the CPU 110 controls each unit of the controller 100 by executing a program stored in the memory 120.

  The touch panel 130 includes a tablet 132 and a display 131 laid on the surface of the tablet 132. As will be described later, when the radio controlled car 200 includes a 3D camera, the display 131 is preferably a 3D display.

  The touch panel 130 may be any type such as a resistance film method, a surface acoustic wave method, an infrared method, an electromagnetic induction method, and a capacitance method. The touch panel 130 may include an optical sensor liquid crystal. The touch panel 130 (tablet 132) detects a touch operation on the touch panel 130 by an external object every predetermined time, and inputs the touch coordinates (touch position) to the CPU 110. In other words, the CPU 110 sequentially acquires touch coordinates from the touch panel 130.

  The speaker 140 outputs sound based on a command from the CPU 110. For example, CPU 110 causes speaker 140 to output sound based on the sound data.

  The button 150 is disposed on the surface of the controller 100. A plurality of buttons such as a direction key, a determination key, and a numeric keypad may be arranged on the controller 100. The button 150 receives a command from the user. The button 150 inputs a command from the user to the CPU 110.

  The memory interface 160 reads data from the external storage medium 161. In other words, the CPU 110 reads data stored in the external storage medium 161 via the memory interface 160 and stores the data in the memory 120. Conversely, the CPU 110 reads data from the memory 120 and stores the data in the external storage medium 161 via the memory interface 160.

  The storage medium 161 includes a CD (Compact Disc), a DVD (Digital Versatile Disk), a BD (Blu-ray Disc), a USB (Universal Serial Bus) memory, a memory card, an FD (Flexible Disk), a hard disk, and a magnetic tape. , Cassette tape, MO (Magnetic Optical Disc), MD (Mini Disc), IC (Integrated Circuit) card (excluding memory card), optical card, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory) And a medium for storing the program.

  The communication interface 170 is realized by an antenna or a connector. The communication interface 170 exchanges data with the radio controlled car 200 and other controllers 100 by wireless communication. CPU 110 receives a program, map data, and the like from another computer via communication interface 170. The CPU 110 transmits course data to the other controller 100 and receives course data from the other controller 100 via the communication interface 170. The CPU 110 transmits a movement command to the radio controlled car 200 via the communication interface 170, and receives a captured image, a current position, and a current orientation from the radio controlled car 200.

The clock 180 measures time or a period based on a command from the CPU 110.
The CPU 110 controls each unit of the controller 100 by executing a program stored in the memory 120 or the storage medium 161. For example, the CPU 110 executes a control process of the radio controlled car 200 by executing a program stored in the memory 120 or the storage medium 161.

  Specifically, CPU 110 receives a captured image from radio controlled car 200 via communication interface 170. CPU 110 displays the captured image on touch panel 130. CPU 110 accepts a command (movement command) for moving radio controlled car 200 from the user via touch panel 130 or button 150. CPU 110 transmits a movement command to radio controlled car 200 via communication interface 170.

  CPU 110 receives the current position and orientation from radio controlled car 200 via communication interface 170. CPU 110 stores time series data of the current position of radio controlled car 200 in memory 120.

  CPU110 acquires the locus | trajectory 301 of the radio controlled car 200 based on time series data. The CPU 110 creates data indicating the course 302 for the radio controlled car 200 based on the trajectory 301 of the radio controlled car 200.

  Data indicating a course (also referred to as course data) includes 3D objects (model data) such as a white line 301X indicating a center line of the course, a white line 302X indicating an end line of the course, and a course pylon 303X. The CPU 110 associates the course position, shape, and orientation with actual map data. Alternatively, the CPU 110 associates the position, shape, and direction of the course with latitude / longitude.

  Alternatively, the CPU 110 may create the course data without causing the radio controlled car 200 to travel. More specifically, CPU 110 according to the present embodiment accepts a map acquisition command from the user via touch panel 130 or button 150. The CPU 110 downloads map data from an external server or the like via the communication interface 170. The CPU 110 may read map data from the memory 120 or the storage medium 161.

  CPU 110 accepts a command for creating a course for radio controlled car 200 from the user via touch panel 130 or button 150. FIG. 3 is an image diagram showing a state in which a user according to the present embodiment inputs a command for creating a course for radio controlled car 200.

  Referring to FIG. 3, CPU 110 accepts a slide operation by the user while displaying a map image on touch panel 130. For example, the CPU 110 obtains the finger trajectory 301 on the map image by sequentially obtaining the touch position of the finger on the map image via the touch panel 130.

  Alternatively, the CPU 110 displays a map image and a pointer on the touch panel 130. The CPU 110 receives a pointer movement command from the user via the button 150. CPU110 acquires the locus | trajectory 301 of the pointer on a map image by acquiring the position with respect to the map image of a pointer sequentially. For example, the user moves the pointer by operating a direction key while pressing a predetermined key.

  The CPU 110 creates data indicating the course 302 for the radio controlled car 200 based on the finger trajectory 301. The CPU 110 creates a course pylon, a track line, etc. for indicating the end of the course on both sides of the trajectory 301.

  Alternatively, the controller 100 may store a plurality of course data in advance. That is, the user may select a desired course from a plurality of courses prepared in advance. In this case, for example, the position, shape, and orientation of the course are associated with map data or latitude / longitude in advance. Alternatively, only the shape of the course is prepared, and the position and orientation of the course may be designated when the user selects the course.

  In the present embodiment, CPU 110 transmits course data to other controllers 100 and receives course data from other controllers 100. The CPU 110 stores the course data received from the other controller 100 in the memory 120. CPU 110 accepts selection of the course data from the user via touch panel 130 or button 150.

  For example, a user of the first and second controllers 100X and 100Y can race by running the first and second radio controlled cars 200X and 200Y simultaneously on the same course.

  After the course is determined, the CPU 110 starts a race based on a user instruction. The CPU 110 acquires the position and orientation of the radio controlled car 200 in the course based on the course data and the current position and orientation of the radio controlled car 200.

  CPU110 creates the virtual image which shows the 3D object seen from the radio controlled car 200 from the data of the course based on the position and direction of the radio controlled car 200 in the course. CPU110 superimposes and displays a virtual image on the picked-up image of radio controlled car 200. FIG. For example, the radio controlled car 200 of another controller 100 may be reflected in the captured image.

  FIG. 4 is an image diagram showing controller 100 in the normal mode and the race mode according to the present embodiment. More specifically, FIG. 4A is an image diagram showing the touch panel 130 of the controller 100 before the race starts or when the radio controlled car 200 is running to create a course. FIG. 4B is an image diagram showing the touch panel 130 of the controller 100 during the race.

  Referring to FIG. 4A, CPU 110 causes touch panel 130 to display a photographed image from radio controlled car 200 before the race starts or when radio controlled car 200 is running to create a course. . Referring to FIG. 4B, when the race starts or during the race, the CPU 110 displays a white line 301X indicating the center of the course, a white line 302X indicating the end of the course, on the photographed image from the radio controlled car 200. A virtual image such as the Cospylon 303X is superimposed.

  More specifically, the CPU 110 may superimpose a 2D or 3D virtual image suitable for starting a race on the captured image on the touch panel 130. That is, the memory 120 stores a start virtual image. FIG. 5 is an image diagram showing the controller 100 immediately before the start of the race according to the present embodiment.

  Referring to FIG. 5, CPU 110 receives a start point and a goal point in the course via touch panel 130 or button 150. CPU110 reads a virtual image from the memory 120, when a race is started based on the present position and direction of the radio controlled car 200 in a course. CPU 110 causes touch panel 130 to display a captured image, white lines 301X and 302X, coarse pylon 303X, characters indicating the start timing, an image of a traffic light, and the like.

  Further, the CPU 110 may cause the touch panel 130 to display a 2D or 3D virtual image suitable for the race goal on the captured image. That is, the memory 120 stores a goal virtual image. FIG. 6 is an image diagram showing the controller 100 when the radio controlled car 200 according to the present embodiment is located immediately before the goal point of the race.

  The CPU 110 receives a start point and a goal point in the course via the touch panel 130 or the button 150. CPU 110 accepts designation of how many weeks the course is to be performed via touch panel 130 or button 150.

  Referring to FIG. 6, CPU 110 reads a virtual image from memory 120 when radio controlled car 200 approaches the goal point based on the current position and orientation of radio controlled car 200 on the course. CPU 110 causes touch panel 130 to display a captured image, white lines 301X and 302X, course pylon 303X, an image showing a goal point, and the like.

<Configuration of radio controlled car 200>
Next, an aspect of a specific configuration of the radio controlled car 200 will be described. FIG. 7 is a block diagram showing a hardware configuration of radio controlled car 200 according to the present embodiment. Referring to FIG. 7, the radio controlled car 200 includes, as main components, a CPU 210, a memory 220, a moving mechanism 230, a GPS 240, an electronic compass 250, a memory interface 260, a communication interface 270, a watch 280, and the like. , And camera 290.

  The memory 220 is realized by various RAMs, ROMs, hard disks, and the like. The memory 220 stores a program executed by the CPU 210 and the like. In other words, the CPU 210 controls each part of the radio controlled car 200 by executing a program stored in the memory 220.

  The moving mechanism 230 moves the radio controlled car 200 based on a command from the CPU 210. For example, the moving mechanism 230 includes a motor, a shaft, a tire, and the like. However, the moving mechanism 230 may be a propeller, a wing, a leg, or the like. The moving mechanism 230 moves the radio controlled car 200 in accordance with a moving command from the controller 100.

  The GPS 240 acquires the current position of the radio controlled car 200. The current position is transmitted to the controller 100 via the communication interface 270.

  The electronic compass 250 acquires the direction of the radio controlled car 200. The direction is transmitted to the controller 100 via the communication interface 270.

  The memory interface 260 reads data from the external storage medium 261. In other words, the CPU 210 reads data stored in the external storage medium 261 via the memory interface 260 and stores the data in the memory 220. Conversely, the CPU 210 reads data from the memory 220 and stores the data in the external storage medium 261 via the memory interface 260.

  Since the storage medium 261 is implemented in the same manner as the storage medium 161 of the controller 100, description thereof will not be repeated here.

  The communication interface 270 is realized by an antenna or a connector. The communication interface 270 exchanges data with the controller 100 by wireless communication. In other words, the CPU 210 receives a movement command from the controller 100 via the communication interface 270, and transmits a captured image, the current position, and the current orientation.

The clock 280 measures time or a period based on a command from the CPU 210.
The camera 290 is disposed at the front part of the radio controlled car 200. The camera 290 captures a scene in front of the radio controlled car 200. The camera 290 preferably includes a right camera for capturing a right-eye image and a left camera for capturing a left-eye image. That is, it is preferable that the camera 290 can capture a 3D image.

  The CPU 210 controls each unit of the radio controlled car 200 by executing a program stored in the memory 220 or the storage medium 261. For example, the CPU 210 executes a program stored in the memory 220 or the storage medium 261.

  More specifically, CPU 210 according to the present embodiment sequentially transmits images taken by camera 290 to controller 100 via communication interface 270. The CPU 210 transmits the current position and orientation of the radio controlled car 200 to the controller 100 via the communication interface 270 in response to a request from the controller 100 or periodically. The CPU 210 receives a movement command from the controller 100 via the communication interface 270. The CPU 210 drives the movement mechanism 230 based on the movement command.

  For example, the CPU 210 receives a forward command, a reverse command, a direction change command, an acceleration / deceleration command, and the like as a movement command via the communication interface 270. The CPU 210 drives the moving mechanism 230 based on the command to move the radio controlled car 200 forward, backward, change direction, or increase the speed.

<Control method>
Below, the process performed with the controller 100 which concerns on this Embodiment is mainly demonstrated. That is, a method for controlling radio controlled car 200 in controller 100 according to the present embodiment will be described. FIG. 8 is a sequence diagram illustrating a control method in the controller 100 according to the first embodiment.

  Referring to FIG. 8, when CPU 110 of controller 100 receives a race start command from the user via touch panel 130 or button 150, CPU 110 executes the processing from step S <b> 106. In the following, it is assumed that the CPU 110 has received the current position and orientation from the radio controlled car 200 once via the communication interface 170.

  CPU 110 creates course 3D virtual data by reading course data from memory 120 (step S106). CPU 110 determines whether a captured image or a current position and orientation is received from radio controlled car 200 via communication interface 170 (step S108). CPU110 repeats the process of step S108, when a picked-up image or the present position and direction are not received from the radio controlled car 200 (when it is NO in step S108).

  When CPU 110 receives the captured image and the current position and orientation from radio controlled car 200 (YES in step S108), CPU 110 views the 3D virtual model indicating the course based on the current position and orientation of the latest radio controlled car 200. Is determined (step S112). More specifically, the CPU 110 determines the viewpoint for the 3D virtual model based on the position and orientation in the course.

  CPU 110 creates a 3D virtual image from the determined viewpoint (step S114). CPU110 superimposes the newest 3D virtual image created on the latest captured image (step S116). CPU 110 causes touch panel 130 to display a composite image of the latest captured image and the latest 3D virtual image (step S118).

  CPU 110 determines whether an instruction for ending the race is received from the user via touch panel 130 or button 150 (step S120). CPU110 repeats the process from step S108, when the command for ending a race is not received (when it is NO in step S120). CPU110 complete | finishes the control process of the radio controlled car 200, when the command for complete | finishing a race is received (when it is YES in step S120).

[Embodiment 2]
Next, a second embodiment of the present invention will be described. In the network system 1 according to the present embodiment, the first controller 100X displays a virtual image indicating the second radio controlled car 200Y that has already traveled on the course during the race of the first radio controlled car 200X. .

  Hereinafter, the description of the same configuration as that of network system 1 according to Embodiment 1 will not be repeated.

<Overview of network system operation>
First, an outline of the operation of the network system 1 according to the present embodiment will be described.

  Referring to FIG. 1, network system 1 includes controllers 100X and 100Y and radio controlled cars 200X and 200Y. However, in the present embodiment, it is assumed that the first radio controlled car 200X travels after the travel of the second radio controlled car 200Y ends. Hereinafter, the controllers 100X and 100Y are collectively referred to as the controller 100 for the sake of explanation. The radio controlled cars 200X and 200Y are collectively referred to as a radio controlled car 200.

  The user places the radio controlled car 200 on the ground of a park or garden. The user controls the operation of the radio controlled car 200 via the controller 100.

  The radio controlled car 200 is equipped with a camera for photographing the front of the radio controlled car 200. The radio controlled car 200 transmits captured images to the controller 100 sequentially.

  The radio controlled car 200 is equipped with a GPS for measuring the current position of the radio controlled car 200. The radio controlled car 200 sequentially transmits the current position to the controller 100.

  The radio controlled car 200 is equipped with an electronic compass for measuring the current direction (attitude) of the radio controlled car 200. The radio controlled car 200 transmits the current direction to the controller 100 sequentially.

  The controller 100 displays a captured image from the radio controlled car 200. The user inputs a command (movement command) for moving the radio controlled car 200 to the controller 100 while viewing the captured image. The controller 100 transmits a movement command from the user to the radio controlled car 200.

  In the present embodiment, the controller 100 accumulates time series data (course creation time series data) of the current position and direction from the radio controlled car 200. The controller 100 acquires the trajectory 301 of the radio controlled car 200 based on the time series data. The controller 100 creates data indicating the course 302 for the radio controlled car 200 based on the trajectory 301 of the radio controlled car 200.

  The data indicating the course includes a 3D object such as a white line indicating a course track or a course pylon. The course position, shape, and orientation are associated with actual map data. Alternatively, the course position, shape, and orientation are associated with latitude and longitude.

  However, the controller 100 may accept a course creation command via the touch panel 130 or the like. For example, the controller 100 receives a slide operation on the map from the user while the map is displayed. The controller 100 determines the position, shape, and orientation of the course based on the slide operation.

  Alternatively, the controller 100 may store a plurality of course data in advance. That is, even if the radio controlled car 200 does not run, the user may select a desired course from a plurality of courses prepared in advance. In this case, for example, the position, shape, and orientation of the course are associated with map data or latitude / longitude in advance. Alternatively, only the shape of the course is prepared, and when the user selects a course, the position and orientation of the course may be designated by the user.

  In the present embodiment, the first controller 100X for controlling the first radio controlled car 200X can communicate with the second controller 100Y for controlling the second radio controlled car 200Y. After the course is determined, the first controller 100X transmits the course data to the second controller 100Y or receives the course data from the second radio controlled car 200Y. That is, the first controller 100X and the second controller 100Y can control the traveling of the radio controlled cars 200X and 200Y based on the common course data.

  For example, when the second controller 100Y travels the course first, the second controller 100Y accumulates time series data (race time series data) of the position and orientation of the second radio controlled car 200Y. The first controller 100X acquires time-series data of the position and orientation of the radio controlled car 200Y controlled by the second controller 100Y before running the first radio controlled car 200X.

  After acquiring the time series data, the controller 100 starts a race based on a user command. The controller 100 acquires the position and orientation of the radio controlled car 200 in the course based on the course data and the current position and orientation of the radio controlled car 200. The controller 100 refers to time series data of another radio controlled car from the other controller 100, and acquires the position and orientation of the other radio controlled car on the course based on the time from the start of the race.

  Based on the position and orientation of the radio controlled car 200 in the course, the controller 100 creates a virtual image indicating the 3D object viewed from the radio controlled car 200 from the course data and the positions and orientations of other radio controlled cars. In the present embodiment, the virtual image includes not only an image showing a course but also an image showing another radio controlled car. Examples of other radio controlled car images include pre-registered sports car animations and face pictures of other radio controlled cars.

  The controller 100 displays the virtual image superimposed on the captured image of the radio controlled car 200.

  While the first radio controlled car 200X is traveling on the course, the first controller 100X accumulates time series data (race time series data) of the position and orientation of the first radio controlled car 200X. When the traveling of the first radio controlled car 200X ends, the first controller 100X transmits time series data of the position and orientation of the first radio controlled car 200X to the second controller 100Y or the third controller.

  As described above, in the network system 1 according to the present embodiment, the user looks at the virtual image showing the virtual course and the other radio controlled car, while looking at the radio controlled car 200 controlled by the user, The movement can be controlled. As a result, the user can make the control of the movement of the radio controlled car 200 full of realism.

  Hereinafter, a specific configuration of the network system 1 for realizing such a function will be described in detail.

<Configuration of Controller 100>
Since the hardware configuration of controller 100 is the same as that of the first embodiment, description thereof will not be repeated here. Hereinafter, data stored in the memory 120 and the operation of the CPU 110 of the controller 100 will be described.

  The CPU 110 controls each unit of the controller 100 by executing a program stored in the memory 120 or the storage medium 161. For example, the CPU 110 executes a control process of the radio controlled car 200 by executing a program stored in the memory 120 or the storage medium 161.

  Specifically, CPU 110 receives a captured image from radio controlled car 200 via communication interface 170. CPU 110 displays the captured image on touch panel 130. CPU 110 accepts a command (movement command) for moving radio controlled car 200 from the user via touch panel 130 or button 150. CPU 110 transmits a movement command to radio controlled car 200 via communication interface 170.

  CPU 110 receives the current position and orientation from radio controlled car 200 via communication interface 170. CPU 110 stores time series data (course creation time series data) of the current position and direction of radio controlled car 200 in memory 120.

  CPU110 acquires locus 301 of radio controlled car 200 based on time series data for course creation. The CPU 110 creates data indicating the course 302 for the radio controlled car 200 based on the trajectory 301 of the radio controlled car 200.

  Data indicating a course (also referred to as course data) includes a 3D object (model data) such as a white line indicating a course track or a course pylon. The CPU 110 associates the course position, shape, and orientation with actual map data. Alternatively, the CPU 110 associates the position, shape, and direction of the course with latitude / longitude.

  Alternatively, the CPU 110 may create the course data without causing the radio controlled car 200 to travel. More specifically, CPU 110 according to the present embodiment accepts a map acquisition command from the user via touch panel 130 or button 150. The CPU 110 downloads map data from an external server or the like via the communication interface 170. The CPU 110 may read map data from the memory 120 or the storage medium 161.

  CPU 110 accepts a command for creating a course for radio controlled car 200 from the user via touch panel 130 or button 150. Referring to FIG. 3, CPU 110 accepts a slide operation by the user while displaying a map image on touch panel 130. For example, the CPU 110 obtains the finger trajectory 301 on the map image by sequentially obtaining the touch position of the finger on the map image via the touch panel 130.

  Alternatively, the CPU 110 displays a map image and a pointer on the touch panel 130. The CPU 110 receives a pointer movement command from the user via the button 150. CPU110 acquires the locus | trajectory 301 of the pointer on a map image by acquiring the position with respect to the map image of a pointer sequentially. For example, the user moves the pointer by operating a direction key while pressing a predetermined key.

  The CPU 110 creates data indicating the course 302 for the radio controlled car 200 based on the finger trajectory 301. The CPU 110 creates a course pylon, a track line, etc. for indicating the end of the course on both sides of the trajectory 301.

  Alternatively, the controller 100 may store a plurality of course data in advance. That is, the user may select a desired course from a plurality of courses prepared in advance. In this case, for example, the position, shape, and orientation of the course are associated with map data or latitude / longitude in advance. Alternatively, only the shape of the course is prepared, and the position and orientation of the course may be designated when the user selects the course.

  In the present embodiment, CPU 110 transmits course data to other controllers 100 and receives course data from other controllers 100. The CPU 110 stores the course data received from the other controller 100 in the memory 120. CPU 110 accepts selection of the course data from the user via touch panel 130 or button 150.

  After the course is determined and the user of the other controller 100 runs another radio controlled car on the course, the CPU 110 receives the position and orientation of the other radio controlled car from the other controller 100 via the communication interface 170. Series data (race time series data) is received. CPU 110 stores in memory 120 time-series data of the positions and orientations of other radio controlled cars.

  FIG. 9 is an image diagram showing a database 123 of race time-series data of positions and orientations of other radio controlled cars stored in the memory 120 according to the second embodiment. Referring to FIG. 9, CPU 110 associates the position and orientation of another radio controlled car transmitted from other controller 100 with the time from the start of the race in which the position and orientation are measured, and at the time of racing. It is stored in the database 123 as series data. In the present embodiment, the controller 100 stores in the database 123 time series data for racing at the position of its own radio controlled car 200 and time series data for racing at the positions and orientations of the other two radio controlled cars.

  More specifically, regarding the elapsed time (HH: MM: SS), in the case of a radio-controlled device that moves at high speed like an airplane, the CPU 110 measures and records the time up to 1/100 second, and the time is recorded as HH: Stored in the database 123 in the format of MM: SS: NN. Regarding the position coordinates, it is desirable to use a data format that is standardly used in the field of GPS. N: north latitude, S: south latitude, E: east longitude, W: west longitude. Regarding the direction (attitude), it is desirable to use a data format that is standardly used in GPS, electronic compass, and the like. Note that 0: north, 90: east, 180: south, 270: west.

  Based on the user's command, the CPU 110 starts the race. The CPU 110 acquires the position and orientation of the radio controlled car 200 in the course based on the course data and the current position and orientation of the radio controlled car 200. CPU110 acquires the position and direction of the other radio controlled car in a course based on the time from a race start by referring to the time series data of the other radio controlled car of database 123.

  Based on the position and orientation of the radio controlled car 200 in the course, the CPU 110 creates a virtual image indicating the 3D object viewed from the radio controlled car 200 from the course data and the positions and orientations of the other radio controlled cars. CPU 110 superimposes the virtual image on the captured image of radio controlled car 200 and displays it on touch panel 130.

  For example, when another radio controlled car precedes, the CPU 110 includes the image of the other radio controlled car in the virtual image. Examples of other radio controlled car images include pre-registered sports car animations and face pictures of other radio controlled cars.

  While the radio controlled car 200 is traveling on the course, the CPU 110 accumulates the race time series data of the position and orientation of the radio controlled car 200 in the database 123 of the memory 120. When the traveling of the radio controlled car 200 is completed, the CPU 110 transmits the time series data for the race of the position and direction of the (radio) controlled car 200 to the other controller 100 via the communication interface 170.

  That is, also in the present embodiment, referring to FIG. 4A, the CPU 110 starts from the radio controlled car 200 before the race starts or when the radio controlled car 200 is running to create a course. Is displayed on the touch panel 130. Referring to FIG. 4B, when the race starts or during the race, the CPU 110 displays a white line 301X indicating the center of the course, a white line 302X indicating the end of the course, on the photographed image from the radio controlled car 200. A virtual image such as the Cospylon 303X is superimposed.

  Further, the CPU 110 may superimpose a 2D or 3D virtual image suitable for starting a race on the captured image on the touch panel 130. That is, the memory 120 stores a start virtual image.

  Referring to FIG. 5, CPU 110 receives a start point and a goal point in the course via touch panel 130 or button 150. CPU110 reads a virtual image from the memory 120, when a race is started based on the present position and direction of the radio controlled car 200 in a course. CPU 110 causes touch panel 130 to display a captured image, white lines 301X and 302X, coarse pylon 303X, characters indicating the start timing, an image of a traffic light, and the like.

  Further, the CPU 110 may cause the touch panel 130 to display a 2D or 3D virtual image suitable for the race goal on the captured image. That is, the memory 120 stores a goal virtual image.

  The CPU 110 receives a start point and a goal point in the course via the touch panel 130 or the button 150. CPU 110 accepts designation of how many weeks the course is to be performed via touch panel 130 or button 150.

  Referring to FIG. 6, CPU 110 reads a virtual image from memory 120 when radio controlled car 200 approaches the goal point based on the current position and orientation of radio controlled car 200 on the course. CPU 110 causes touch panel 130 to display a captured image, white lines 301X and 302X, course pylon 303X, an image showing a goal point, and the like.

<Configuration of radio controlled car 200>
Note that the hardware configuration of the radio controlled car 200 is the same as that of the first embodiment, and therefore description thereof will not be repeated here.

<Control method>
Below, the process performed with the controller 100 which concerns on this Embodiment is mainly demonstrated. That is, a method for controlling radio controlled car 200 in controller 100 according to the present embodiment will be described. FIG. 10 is a sequence diagram illustrating a control method in the controller 100 according to the second embodiment.

  Referring to FIG. 10, when CPU 110 of controller 100 receives a race start command from the user via touch panel 130 or button 150, it executes the processing from step S <b> 202. In the following, it is assumed that the CPU 110 has received the current position and orientation from the radio controlled car 200 once via the communication interface 170. Further, it is assumed that the CPU 110 stores the time series data for the race of the position and direction of the other radio controlled car controlled by the other controller 100 via the communication interface 170 in the database 123.

  CPU 110 uses clock 180 to start measuring the time from the start of the race (step S202). The CPU 110 refers to the clock 180 and acquires the elapsed time from the start of the race (step S204). The CPU 110 creates the 3D virtual data of the course by reading the course data from the memory 120 (step S206).

  More specifically, in the present embodiment, the CPU 110 refers to the time series data of other radio controlled cars in the database 123, and based on the time from the start of the race, the position and orientation of the other radio controlled cars on the course. To get. CPU 110 creates 3D virtual data including the course and the other radio controlled car from the data of the course and the position and orientation of the other radio controlled car.

  CPU 110 determines whether a captured image or a current position and orientation are received from radio controlled car 200 via communication interface 170 (step S208). CPU110 repeats the process from step S204, when a captured image or the present position and direction are not received from the radio controlled car 200 (when it is NO in step S208).

  When CPU 110 receives the captured image and the current position and direction from radio controlled car 200 (YES in step S208), CPU 110 stores the current position and direction in database 123 in association with the time from the start of the race (step S208). S210). CPU110 determines the viewpoint with respect to 3D virtual model which shows a course and another radio controlled car based on the present position and direction of the newest radio controlled car 200 (step S212). That is, CPU110 determines the viewpoint with respect to 3D virtual model based on the position and direction of the own radio controlled car 200 with respect to a course and another radio controlled car.

  The CPU 110 creates a 3D virtual image from the determined viewpoint (step S214). CPU110 superimposes the newest 3D virtual image produced on the newest captured image (step S216). CPU 110 causes touch panel 130 to display a composite image of the latest captured image and the latest 3D virtual image (step S218).

  CPU 110 determines whether an instruction for ending the race has been received from the user via touch panel 130 or button 150 (step S220). CPU110 repeats the process from step S204, when the command for ending a race is not received (when it is NO in step S220). CPU110 complete | finishes measurement of time, when the command for complete | finishing a race is received (when it is YES in step S220) (step S222). CPU110 complete | finishes the control process of the radio controlled car 200. FIG.

<Other application examples>
It goes without saying that the present invention can also be applied to a case where it is achieved by supplying a program to a system or apparatus. Then, external storage media 161, 261 and memories 120, 220 storing programs represented by software for achieving the present invention are supplied to the system or apparatus, and the computer (or CPU or MPU of the system or apparatus) is supplied. ) Can also read and execute the program code stored in the external storage medium 161, 261 or the memory 120, 220, so that the effects of the present invention can be enjoyed.

  In this case, the program code itself read from the external storage media 161 and 261 and the memories 120 and 220 realizes the functions of the above-described embodiment, and the external storage media 161 and 261 storing the program code are provided. The memories 120 and 220 constitute the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code However, it is needless to say that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Furthermore, the program code read from the external storage medium 161, 261 or the memory 120, 220 is written to another storage medium provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiment may be realized by the processing. Needless to say, it is included.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  1 network system, 100 controller, 110 CPU, 120 memory, 130 touch panel, 131 display, 132 tablet, 140 speaker, 150 button, 160 memory interface, 161 storage medium, 170 communication interface, 180 clock, 200 radio controlled car, 210 CPU, 220 memory, 230 moving mechanism, 240 GPS, 250 electronic compass, 260 memory interface, 261 storage medium, 270 communication interface, 280 clock, 290 camera, 301 locus, 302 course, 301X white line indicating the center of the course, end of 302X course White line indicating 303X corspylon.

Claims (10)

A network system comprising a model device and a controller,
The model device is
A moving unit for moving the model apparatus based on a command from the controller;
A camera for photographing the front of the model device;
A GPS unit for obtaining the position of the model device;
A compass for obtaining the orientation of the model device;
A first communication interface for receiving the command from the controller and transmitting a captured image from the camera, the position, and the orientation to the controller;
The controller is
Display,
An operation unit for receiving the command;
A second communication interface for communicating with the model device;
A memory for storing the course data for the model device;
Based on the position and orientation received from the model device, a virtual image for display is created from the data of the course, and an image obtained by combining the captured image and the virtual image received from the model device is A network system including a processor for displaying on a display. The processor is
The network system according to claim 1, wherein course data for the model device along a locus of the position is created based on time-series data of the position received from the model device. The memory stores a plurality of course data for the model apparatus,
The network system according to claim 1, wherein the processor receives a course selection command for the plurality of model devices via the operation unit.   4. The network system according to claim 1, wherein the processor transmits course data for the model device to another controller via the second communication interface. 5.   5. The network system according to claim 1, wherein the processor receives course data for the model device from another controller via the second communication interface. 6. The processor is
Receiving time-series data in which the position of another model device and the elapsed time from the start time are associated with each other from another controller via the second communication interface;
Obtaining the position of the other model device corresponding to the elapsed time from the start time of the model device from the time series data,
The image indicating the other model device is added to the virtual image for display based on the position and the direction received from the model device and the position of the other model device. The network system according to any one of the above. A control method in a network system comprising a model device and a controller for storing course data for the model device,
Receiving a command for moving the model device by the controller;
The controller transmitting the command to the model device;
The model device moving the model device based on a command from the controller;
The model device, photographing the front of the model device;
The model device acquires a position of the model device;
The model device acquires the orientation of the model device; and
The model device transmits a captured image, the position, and the orientation to the controller;
The controller creates a virtual image for display from the course data for the model device based on the position and the orientation received from the model device;
The controller includes a step of causing a display to display an image obtained by combining the captured image received from the model device and the virtual image. Display,
A memory for storing the course data for the model device;
An operation unit for receiving a command for moving the model device;
A communication interface for transmitting the command to the model device and receiving a photographed image and the position and orientation of the model device from the model device;
Based on the position and orientation received from the model device, a virtual image for display is created from the data of the course, and an image obtained by combining the captured image and the virtual image received from the model device is And a processor for displaying on a display. A control method in a controller including a memory for storing course data for a model device, a display, a communication interface, and a processor,
Receiving a command for the processor to move the model device;
The processor transmitting the command to the model device via the communication interface;
Receiving the photographed image and the position and orientation of the model device from the model device via the communication interface;
The processor creates a virtual image for display from the course data based on the position and the orientation received from the model device;
A control method comprising: a step of causing the display to display an image obtained by combining the captured image received from the model device and the virtual image. A control program for causing a controller including a memory for storing course data for a model device, a display, a communication interface, and a processor to control the model device,
Receiving a command for moving the model device;
Transmitting the command to the model device via the communication interface;
Receiving the photographed image and the position and orientation of the model device from the model device via the communication interface;
Creating a virtual image for display from the course data based on the position and orientation received from the model device;
The control program which performs the step which displays the image which synthesize | combined the said picked-up image received from the said model apparatus and the said virtual image on the said display.

Images