JP2023128653A - Remote sightseeing system, service providing device, autonomous movable body, and service providing method - Google Patents

Abstract

To control a movable body in real time according to change in a situation even when a user is allowed to operate the movable body.SOLUTION: According to one embodiment of the present invention, there is provided a remote sightseeing system that comprises: a setting part which sets an inhibited area that an autonomous movable body is inhibited from entering; an operation part which receives user's operations; and a control part which controls a movement of the autonomous movable body based upon the operations received by the operation part when the autonomous movable body is moving other than the inhibited area set by the setting part, and restricts the autonomous movable body from moving when an operation to enter the inhibited area set by the setting part is received.SELECTED DRAWING: Figure 1

Description

The present invention relates to a remote tourism system, a service providing device, an autonomous mobile object, and a service providing method.

Conventionally, techniques for controlling the movement of robots are known. As this type of technology, for example, a control device described in Patent Document 1 is known. When a person is detected between the automatically operating robot and the destination, the control device of Patent Document 1 prevents persons whose behavior is difficult to predict, such as elderly people and children, from entering a dangerous area. controlling the robot. The moving object system described in Patent Document 2 sets a no-go area on a map, searches for a route to the destination, and automatically moves the robot so as not to enter the set no-go area. .

Japanese Patent Application Publication No. 2011-200947 JP2021-64147A

For example, there is a need for elderly people to enjoy sightseeing by operating robots, and there is a need to increase the satisfaction level of sightseeing by allowing elderly people to manually operate robots. Furthermore, it is required to control the robot's movement in real time in response to operational errors by the elderly and changes in the surrounding conditions of the robot, such as the weather at tourist spots, crowds, and obstacles on the route. However, since the control device of Patent Document 1 and the moving object system of Patent Document 2 described above are technologies for automatically controlling a moving robot, they may not be able to meet the above requirements.

The present disclosure has been made in view of these circumstances, and provides a remote tourism system and service that can control a moving object such as a robot in real time according to changes in the situation even when the user operates the moving object. The purpose is to provide a providing device, an autonomous mobile object, and a service providing method.

The present disclosure has been made to solve the above-mentioned problems, and one aspect of the present disclosure includes a setting unit that sets a prohibited area in which autonomous mobile objects are prohibited from entering a tourist area, and an operation unit that accepts user operations. and when the autonomous mobile body is moving outside the prohibited area set by the setting unit, the movement of the autonomous mobile body is controlled based on the operation received by the operation unit, and the setting unit controls the movement of the autonomous mobile body based on the operation received by the operation unit. The remote tourism system includes a control unit that restricts movement of the autonomous mobile body when an operation to enter the set prohibited area is received.

Another aspect of the present disclosure includes a setting unit that sets a prohibited area in which autonomous mobile objects are prohibited from entering a tourist area, a communication unit that receives user operation information, and a setting unit that sets the autonomous mobile object to be set by the setting unit. If the autonomous mobile body is moving outside the prohibited area set by the setting unit, the movement of the autonomous mobile body is controlled based on the operation information, and when operation information for entering the prohibited area set by the setting unit is received, the autonomous mobile body A service providing device includes a control unit that limits movement of an autonomous mobile body.

Another aspect of the present disclosure includes a storage unit that stores a prohibited area in which autonomous mobile bodies are prohibited from entering a tourist area, a communication unit that receives user operation information, and a communication unit that stores the autonomous mobile body in the storage unit. When the autonomous mobile body is moving outside the prohibited area, the movement of the autonomous mobile body is controlled based on the operation information, and when an operation to enter the prohibited area stored in the storage unit is received, the autonomous mobile body The autonomous mobile body includes a control unit that limits movement of the mobile body.

Other aspects of the present disclosure include the steps of setting a prohibited area in which autonomous mobile objects are prohibited from entering a tourist area, accepting user operations, and when the autonomous mobile object is moving outside the prohibited area. The service providing method includes the steps of controlling the movement of the autonomous mobile body based on the operation, and restricting the movement of the autonomous mobile body when an operation to enter the prohibited area is received.

According to one aspect of the present invention, even when a user operates a moving object, the moving object can be controlled in real time according to changes in the situation.

It is a block diagram showing an example of a remote sightseeing system in an embodiment. It is a block diagram showing an example of an autonomous mobile object in an embodiment. It is a block diagram showing an example of a sensing part in an embodiment. It is a block diagram showing an example of a service provision device in an embodiment. FIG. 3 is a block diagram for explaining a prohibited area setting unit in the embodiment. FIG. 3 is a diagram showing an example of content in which route information is superimposed on a map in an embodiment. FIG. 3 is a diagram illustrating an example of content including a prohibited area in an embodiment. FIG. 7 is a diagram showing another example of content including a prohibited area in the embodiment. FIG. 7 is a diagram showing another example of content including a prohibited area in the embodiment. FIG. 3 is a diagram for explaining a process of transferring user authority to an autonomous mobile body near a destination in an embodiment. FIG. 3 is a diagram for explaining a process of transferring user authority to an autonomous mobile body corresponding to a specific area in an embodiment. It is a flowchart which shows an example of the process which provides remote tourism service in embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS A remote tourism system, a service providing device, an autonomous mobile object, and a service providing method to which the present invention is applied will be described below with reference to the drawings.

(Configuration of remote tourism system 1)
FIG. 1 is a block diagram showing an example of a remote tourism system according to an embodiment. The remote tourism system 1 includes, for example, one or more autonomous mobile bodies 100, a tourist spot system 200, a user device 300, a digital local currency system 400, a product payment system 500, an authentication system 600, and an operator terminal 700. Equipped with. The autonomous mobile object 100, each part of the sightseeing spot system 200, the user device 300, the digital local currency system 400, the product payment system 500, the authentication system 600, and the operator terminal 700 are connected to a network NW such as the Internet using an NIC (Network It has a communication interface such as an interface card) or a wireless communication module. The network NW may include, for example, a general-purpose network such as the Internet, and a private network such as local 5G or Wi-Fi (registered trademark). The remote tourism system 1 includes, for example, receiving operations from a user such as an elderly person, causing the autonomous mobile body 100 to travel in a tourist area, and providing content including video and audio acquired by the autonomous mobile body 100. provide services; Sightseeing areas include, for example, temples and shrines, historical sites, historic sites, downtown areas, museums, aquariums, theme parks, and the like.

The autonomous mobile object 100 is, for example, a self-propelled robot that can travel in a tourist area according to a user's operation. The tourist attraction system 200 is located at the tourist attraction and on the cloud. The tourist spot system 200 includes, for example, a service providing device 210, a camera system 220, a sensor system 230, and a store system 240. The service providing device 210 performs content providing processing and the like for providing remote tourism services. The camera system 220 includes a plurality of cameras installed in a tourist area that provides remote tourism services. The sensor system 230 includes a plurality of human sensors installed in the tourist area. The store system 240 includes a POS (point of sale) system, a payment system, etc. compatible with stores in tourist areas. The user device 300 is placed, for example, in a user's home. The digital local currency system 400 is, for example, a system for paying electronic currency that can be used within a specific region using a dedicated application on a smartphone owned by a user or a store clerk. The product payment system 500 is, for example, a system for paying for products sold at a store. The authentication system 600 is, for example, a system that performs user authentication and payment for providing remote tourism services. Operator terminal 700 is a personal computer operated by an operator who assists users in providing remote tourism services. The operator performs support activities for the user to smoothly use the remote tourism service through chat, telephone, etc. with the user.

The user device 300 is connected to, for example, a sensor section 310, an operation section 320, a display section 330, and an audio input/output section 340. The sensor unit 310 detects various types of sensor information regarding the user. The sensor unit 310 is, for example, a camera device that captures an image of the user, but is not limited to this, and may include a sensor that detects the user's heartbeat, blood pressure, etc. The operation unit 320 is a device that accepts user operations. The operation unit 320 generates, for example, operation information that instructs the direction of movement of the autonomous mobile body 100. The display unit 330 is a display such as a head mounted display that displays various images. The audio input/output unit 340 is, for example, a microphone and a speaker. The user device 300 includes, for example, a communication section 302, a processing section 304, and a control section 306. The communication section 302 communicates with each section of the remote tourism system 1. The processing unit 304 inputs sensor information, operation information, and audio information, and transmits the input information to each unit of the remote tourism system 1 via the communication unit 302. The control unit 306 provides content received from the tourist spot system 200 by controlling the display unit 330 and the audio input/output unit 340.

The user device 300 may store images captured by a camera device and sounds detected by a microphone as a record of using a remote tourism service, or may store images captured by a camera device and sounds detected by a microphone as a record of using a remote tourism service, or use a social networking service (SNS) etc. to share information with others. It may be sent to a system that provides information sharing services. The user device 300 may, for example, create a composite image in which the user's face photograph taken during remote sightseeing is superimposed with the scenery of the tourist spot, or a composite image in which the scenery of the tourist spot is superimposed on the user's and accompanying person's photos. You can create commemorative photos and post them on SNS. Further, the user device 300 can share photos and the like with fellow users, family members who are not accompanying the user, and the like. Further, the user device 300 may, for example, superimpose a character string representing a sound detected during remote sightseeing on the video imaged by the autonomous mobile body 100 when the sound was detected. As a result, if the user (traveler) and the user's relative's account are linked in the information sharing service, the user device 300 can monitor the tourist's sightseeing by the relative accessing the traveler's account. It is possible to make the user understand the information and take actions such as "like". Furthermore, if the traveler is impressed by the beautiful scenery, some changes in health data such as heartbeat and blood pressure (such as heartbeat faster than usual) appear, so the user device 300 detects changes in the health data. Based on this, a commemorative photograph may be automatically taken and uploaded to SNS or the like. As a result, even if the user is an elderly person or the like and forgets to press the shutter button of the camera, the user can automatically record the information and share it with the family.

FIG. 2 is a block diagram showing an example of the autonomous mobile body 100 in the embodiment. The autonomous mobile body 100 includes, for example, a communication unit 110, a camera unit 120, a sensor unit 130, an audio input/output unit 140, a position acquisition unit 150, a movement control unit 160, a mobile body storage unit 170, and a mobile body storage unit 170. A mechanism 180 is provided. In the embodiment, an autonomous mobile body 100 that runs on a road in a tourist area will be described, but the present invention is not limited to this, and can also be applied to an autonomous mobile body that moves underwater or in the air.

The communication unit 110 communicates with the tourist spot system 200 via the network NW. Note that when prioritizing real-time performance, the communication unit 110 transmits images captured in the tourist area via a communication route (LTE (Long Term Evolution), 4G, 5G, etc.) without going through the tourist spot system 200. It may also be transmitted to the user device 300. The camera unit 120 is, for example, a camera device that captures images around the autonomous mobile body 100 at a 360-degree angle of view. The sensor unit 130 is, for example, a lidar device, and detects objects around the autonomous mobile body 100. The audio input/output unit 140 is, for example, a microphone and a speaker. The position acquisition unit 150 is, for example, a GNSS (Global Navigation Satellite System) receiver, and acquires position information of the autonomous mobile body 100. The moving mechanism 180 includes wheels for moving the autonomous mobile body 100 and a drive mechanism such as a motor.

The movement control unit 160 is realized, for example, by a processor such as a CPU (Central Processing Unit) executing a program stored in a program memory. The movement control unit 160 controls the autonomous mobile body 100 in an integrated manner. The movement control unit 160 controls various communications with the tourist spot system 200 via the communication unit 110. The movement control unit 160 autonomously moves the autonomous mobile body 100 by driving the movement mechanism 180 based on route information, operation information, and prohibited area information. The prohibited area information is information representing a prohibited area that is an area into which the autonomous mobile body 100 is prohibited to enter. The prohibited area includes, for example, private land and rough terrain where it is difficult for the autonomous mobile body 100 to travel. The prohibited area includes a preset prohibited area and a dynamic prohibited area, which will be described later. In the following explanation, the driving mode in which the autonomous mobile object 100 is moved based on the user's operation is referred to as manual operation, and the driving mode in which the autonomous mobile object 100 is automatically moved regardless of the user's operation is referred to as manual driving. Describe it as automatic operation.

The movement control section 160 includes a detection section 162 and a position correction section 164. The detection unit 162 detects an obstacle to the movement of the autonomous mobile body 100. FIG. 3 is a block diagram illustrating an example of a detection unit in the embodiment. The detection unit 162 includes, for example, a feature extraction unit 162a that extracts features of an object, and a recognition unit 162b that recognizes the type of object. The feature extraction unit 162a extracts features from the image captured by the camera unit 120. The recognition unit 162b compares the features extracted by the feature extraction unit 162a with the features stored in the moving object storage unit 170, and recognizes the type of object. The types of objects include types of uneven ground and obstacles on which the autonomous mobile body 100 cannot travel. The recognition unit 162b recognizes, for example, a non-moving object such as a fallen object, garbage, a puddle, fallen leaves, or a tree branch, and the autonomous mobile object 100 detects the object without making a detour based on the size and height of the non-moving object. It is possible to recognize things that cannot be driven. The recognition unit 162b may recognize moving objects such as pedestrians, bicycles, strollers, wheelchairs, and robots, for example. The recognition unit 162b may compare the recognized object with the movement performance information of the autonomous mobile body 100 stored in the mobile body storage unit 170, and determine whether the object is an obstacle to the autonomous mobile body 100. For example, when the level difference that can be overcome by the autonomous mobile body 100 is 5 cm, if the level difference recognized as an object exceeds 5 cm, it is determined that the object is an obstacle to the autonomous mobile body 100, If not, it is determined that the object is not an obstacle to the autonomous mobile body 100. The detection unit 162 detects the range in which the object exists based on the image captured by the camera unit 120 and the signal detected by the sensor unit 130. Thereby, the detection unit 162 detects a prohibited area corresponding to the range of the object. Hereinafter, the prohibited area detected by the detection unit 162 will be referred to as a "dynamic prohibited area," and the information indicating the dynamic prohibited area will be referred to as "dynamic prohibited area information."

The position correction unit 164 compares the image information and position information captured by the tourist area with the image information and position information captured by the autonomous mobile object 100, and based on the comparison result, the position correction unit 164 compares the image information and position information captured by the autonomous mobile object 100. The position of the autonomous mobile body 100 is corrected.

FIG. 4 is a block diagram showing an example of the service providing device 210 in the embodiment. The service providing device 210 is connected to a camera system 220, a sensor system 230, a store system 240, and a storage device 250. The storage device 250 stores, for example, map information, prohibited area information, various contents, object recognition results indicating the location and type of obstacles, route search results, and mobile object management information. The mobile body management information includes, for example, performance information, usage authority information, current position information, movement history information, dynamic prohibited area information, and areas acquired by the autonomous mobile body 100 in correspondence with the identification information of the autonomous mobile body 100. Contains information on the flow of people and obstacles.

The service providing device 210 includes, for example, a communication unit 211, a content providing unit 212, a failure recognition unit 213, a prohibited area setting unit 214, a route search unit 215, a mobile object management unit 216, and a control unit 217. Be prepared. The functional units including the content providing unit 212, the obstacle recognition unit 213, the prohibited area setting unit 214, the route searching unit 215, the mobile object management unit 216, and the control unit 217 are implemented by a processor such as a CPU using a program stored in a program memory. This is achieved by executing. The communication unit 211 communicates with each unit of the remote tourism system 1 including the user device 300, camera system 220, sensor system 230, and store system 240 via the autonomous mobile body 100 and the network NW. For example, the communication unit 211 receives user operation information from the user device 300 and provides the operation information to the autonomous mobile body 100. The communication unit 211 acquires video and audio from the autonomous mobile body 100 and provides content including the acquired video and audio to the user device 300.

The content providing unit 212 provides the user device 300 with various types of content for providing remote tourism services. For example, the content providing unit 212 provides the user device 300 with content in which various information such as a route and a prohibited area are superimposed on video and audio acquired from the autonomous mobile object 100. The failure recognition unit 213 recognizes a failure of the autonomous mobile body 100 in the tourist area based on, for example, image information collected from the camera system 220 and sensor information collected from the sensor system 230. For example, similar to the detection unit 162, the obstacle recognition unit 213 may extract features from the image acquired from the camera system 220 and recognize an object that becomes an obstacle. The prohibited area setting unit 214 sets a prohibited area for the autonomous mobile body 100 in the tourist area. Hereinafter, the prohibited area set by the prohibited area setting unit 214 will be referred to as a "preset prohibited area", and the information representing the preset prohibited area will be referred to as "preset prohibited area information". The preset prohibited area is an area in which the autonomous mobile body 100 cannot run, such as gravel, grass, steep slopes, and steps. The route search unit 215 searches for a route from the current location of the autonomous mobile body 100 to the destination based on map information and preset prohibited area information. The mobile body management unit 216 updates mobile body management information that associates the ID of the autonomous mobile body 100, the user authority granted to the autonomous mobile body 100, and the position of the autonomous mobile body 100, for example. The control unit 217 restricts the movement of the autonomous mobile body 100 when receiving operation information for entering the prohibited area set by the prohibited area setting unit 214. Note that in the embodiment, the movement of the autonomous mobile body 100 can be restricted by both the movement control unit 160 of the autonomous mobile body 100 and the control unit 217 of the service providing device 210, but the present invention is not limited to this. It is only necessary that one of the movement control unit 160 and the control unit 217 can control the movement of the autonomous mobile body 100.

Note that the service providing device 210 causes the autonomous mobile body 100 to move autonomously by transmitting control information for driving the movement mechanism 180 to the autonomous mobile body 100 based on route information, operation information, and prohibited area information. good.

FIG. 5 is a block diagram for explaining the prohibited area setting unit 214 in the embodiment. For example, the prohibited area setting unit 214 collects information for updating the preset prohibited area when the setting timing arrives. The set timing may be, for example, 8 a.m. every day. For example, the prohibited area setting unit 214 uses map information, environmental information, existing preset prohibited area information, camera images of various tourist areas collected by the camera system 220, recognition results by the obstacle recognition unit 213, information collected from the autonomous mobile object 100, etc. The dynamic prohibited area information and the flow of people information or congestion information collected by the sensor system 230 are collected as information for updating the preset prohibited area. The existing preset prohibited area information is the preset prohibited area information updated at the previous setting timing. The map information, environment information, dynamic prohibited area information, and people flow information are, for example, information acquired by the service providing device 210 and accumulated in the storage device 250 during the period from the previous setting timing to the current setting timing. .

The environmental information is, for example, weather information collected from an information providing device (not shown). For example, the prohibited area setting unit 214 determines a prohibited area for the autonomous mobile body 100 in a tourist area based on rainfall information included in weather information. For example, the prohibited area setting unit 214 does not determine that a road surface that easily contains moisture is a prohibited area if the amount of moisture on the road surface is estimated to be less than the standard based on the amount of rainfall. If it is estimated that there are many, it is determined that it is a prohibited area.

The prohibited area setting unit 214 determines whether there is a new obstacle that is not included in the existing preset prohibited area information, based on the camera image, object recognition results, dynamic prohibited area information, and human flow information. . When a new obstacle exists, the prohibited area setting unit 214 updates the preset prohibited area information by adding a new prohibited area to the existing preset prohibited areas. The prohibited area setting unit 214 updates the preset prohibited area information by excluding the preset prohibited area when there is no longer a problem in the existing preset prohibited area. The prohibited area setting unit 214 stores the update result of the preset prohibited area in the storage device 250.

FIG. 6 is a diagram illustrating an example of content in which route information is superimposed on a map according to the embodiment. The route search unit 215 refers to the preset prohibited area information when searching for a route for the autonomous mobile body 100 to reach from the current position to the destination. For example, assume that the preset prohibited area information indicates the area around an obstacle as a preset prohibited area. In this case, since a preset prohibited area due to obstacles exists on the route from the current position to the destination with the shortest distance or time, the route search unit 215 calculates a route that avoids the preset prohibited area. The route search unit 215 may search for a first route that prioritizes the required time and distance to the destination and a second route that includes sightseeing spots other than the destination, as the route for the autonomous mobile body 100 to travel. Note that the route search unit 215 searches for a plurality of second routes with different stopovers, such as sightseeing spots, for example, and the service providing device 210 recommends the plurality of second routes to the user. Thereby, the service providing device 210 may cause the user device 300 to display the first route and the plurality of second routes, and allow the user to select a route. For example, when displaying the first route, the user device 300 displays the first route superimposed on the map, and when displaying the second route, the user device 300 displays tourist spot information in addition to the second route. may be displayed superimposed on the map.

FIG. 7 is a diagram illustrating an example of content including a prohibited area in the embodiment. Assume that the autonomous mobile body 100 is running on a lawn in a park or the like. In this case, as shown in FIG. 7A, the tourist spot system 200 displays a route image indicating the traveling direction of the autonomous mobile body 100 based on the user's operation, and a current user's operation direction in the video including the lawn. Provides content on which operation key images are superimposed. The user device 300 displays content as shown in FIG. 7(a) based on the content provided from the tourist spot system 200. Assume that the user device 300 receives an operation for the autonomous mobile body 100 to approach a prohibited area. This forbidden area may be a preset forbidden area or a dynamic forbidden area. For example, if the distance from the autonomous mobile object 100 to the obstacle measured by the camera image or the sensor unit 130 is shorter than the first threshold (for example, 1 meter), the user device 300 performs the following operation as shown in FIG. 7(b). As shown, content is displayed with a message image superimposed that says "You are approaching gravel (prohibited area)." After that, the user device 300 determines whether the user performs an operation to approach the prohibited area. The user device 300 prevents the autonomous mobile object 100 from entering the prohibited area when the distance from the autonomous mobile object 100 to the obstacle becomes shorter than a second threshold (for example, 50 centimeters) due to the user's operation to approach the prohibited area. The speed or direction of travel of the autonomous mobile body 100 is controlled. Note that the user device 300 may update the operation key image so as to restrict operations that approach the prohibited area. Thereby, the autonomous mobile body 100 can avoid the prohibited area and travel as shown in FIG. 7(c). Note that although the user device 300 displays content including images, the present invention is not limited to this, and the user device 300 may output content including sound or may vibrate the operation unit 320.

FIG. 8 is a diagram showing another example of content including a prohibited area in the embodiment. When the autonomous mobile object 100 is traveling on a road surface without prohibited areas, the user device 300 superimposes a manual route image and an operation key image based on the user's operation on the video, as shown in FIG. 8(a). display the content. As shown in FIG. 8(b), when the autonomous mobile body 100 detects a puddle as a dynamic prohibited area and the autonomous mobile body 100 receives an operation to enter the dynamic prohibited area, the autonomous mobile body 100: Determine whether the distance to bypass the obstacle exceeds a threshold value. If the distance to bypass the obstacle does not exceed the threshold, the autonomous mobile object 100 calculates an automatic route that automatically bypasses the dynamic prohibited area and returns to the manual route. The autonomous mobile body 100 transmits information indicating the automatic route to the service providing device 210, and the service providing device 210 transmits information indicating the automatic route to the user device 300. As a result, the user device 300 displays content in which the automatic route image is superimposed on the video.

As shown in FIG. 8(c), when a dynamic prohibited area is detected on the entire road surface and the autonomous mobile body 100 receives an operation to enter the dynamic prohibited area, the autonomous mobile body 100 detours around the obstacle. Since the distance exceeds the threshold, the vehicle will stop traveling. The autonomous mobile body 100 transmits information notifying that it has stopped traveling and dynamic prohibited area information to the service providing device 210. The service providing device 210 displays content in which a message image, a route re-search button, and a report button are superimposed on the video: "The vehicle cannot run, so the vehicle will stop." Furthermore, the user device 300 updates the operation key image so as to restrict the operation of entering the prohibited area. When the user device 300 receives an operation in which the route re-search button is selected, the user device 300 requests the service providing device 210 to re-search the route. When the user device 300 receives an operation in which the report button is selected, the user device 300 reports to the operator terminal 700 of the remote tourism service, for example. In this way, the autonomous mobile body 100 automatically detours the autonomous mobile body 100 when the distance to bypass the obstacle does not exceed the threshold value, and automatically detours the autonomous mobile body 100 when the distance to bypass the obstacle exceeds the threshold value. It can be stopped.

FIG. 9 is a diagram showing another example of content including a prohibited area in the embodiment. When the autonomous mobile body 100 is traveling on a road surface with no prohibited areas, as shown in FIG. Suppose you recognize it. As shown in FIG. 9(a), the content does not include a vehicle. The autonomous mobile body 100 transmits the recognition result including the vehicle's current position information to the service providing device 210. The service providing device 210 transmits the recognition result to the user device 300. As shown in FIG. 9(b), the user device 300 displays content superimposed on a video, defining an area into which a vehicle may enter based on the recognition result as a dynamic prohibited area. Further, the user device 300 superimposes a message image “Be careful of vehicles” on the video. Further, the user device 300 updates the operation key image so as to restrict operations in a direction approaching the vehicle. Note that it is desirable to recognize not only vehicles but also moving objects such as people and bicycles as objects approaching the autonomous mobile body 100.

FIG. 10 is a diagram for explaining the process of transferring the user's authority to the autonomous mobile body 100 near the destination in the embodiment. When automatically moving the route from the current location to the destination, if there is another autonomous mobile body 100 closer to the destination than the autonomous mobile body 100 for which the user has authority, the service providing device 210 automatically moves the route from the current location to the destination. User authority may be transferred to the autonomous mobile body 100. For example, the service providing device 210 sets a plurality of standby areas in a tourist area in which autonomous mobile bodies 100 to which no user authority has been granted are placed, and when the user authority is released, the autonomous mobile body 100 automatically temporarily move to the waiting area. Thereby, if another autonomous mobile body 100 arrives at a waiting area near the destination after the start of the remote tourism service, the service providing device 210 can transfer the user authority to the other autonomous mobile body 100. .

FIG. 11 is a diagram for explaining a process of transferring user authority to an autonomous mobile body 100 corresponding to a specific area in the embodiment. For example, it is assumed that there is an autonomous mobile object 100 that is suitable for a specific facility such as a museum, a dangerous area such as a cliff or a high place, or a specific area such as a place where a specific activity is performed. In the specific area, for example, there may be a case where the autonomous mobile body 100 cannot travel or a case where the autonomous mobile body 100 is not permitted to take a picture. In this case, the autonomous mobile body 100 to which user authority has been granted may move to a specific area, and the user authority may be transferred to an autonomous mobile body 100 suitable for the specific area. The autonomous mobile object 100 suitable for a specific area can move manually or automatically in the specific area and provide the user device 300 with content including images shot in the specific area. Further, the object to which the user authority is transferred is not limited to the autonomous mobile body 100, but may be a camera device placed in a specific area, a camera worn by a local staff member in the specific area, or the like.

FIG. 12 is a flowchart illustrating an example of processing for providing remote tourism services in the embodiment. The user specifies the date and time and sightseeing area and makes a reservation. At the reserved date and time, the user accesses a website that provides remote tourism services via the user device 300 and selects the reserved tourist area. The user device 300 transmits an authentication request including a user ID and password to the service providing device 210, and the service providing device 210 performs user authentication in cooperation with the authentication system 600 (step S100). If the user authentication is successful, the service providing device 210 grants user authority to the autonomous mobile body 100 and starts remote tourism service. The service providing device 210 determines whether to end the remote tourism service (step S102). If the service providing device 210 does not end the remote tourism service (step S102: NO), the service providing device 210 advances the process to step S104.

The service providing device 210 determines whether a destination is set (step S104). For example, when a destination is set at the time of reservation for a remote tourism service, or when a destination is set based on a user's operation after the start of a remote tourism service, the service providing device 210 is configured to judge. If there is no destination setting, the service providing device 210 advances the process to step S122. If the destination has been set, the service providing device 210 advances the process to step S104 and sets the destination (step S106).

Next, the service providing device 210 searches for a route from the current location of the autonomous mobile body 100 to the destination. The service providing device 210 transmits the content including the searched route information to the user device 300, and causes the user device 300 to display the content (step S108). Note that the service providing device 210 may search for a route with the shortest distance or time from the current location to the destination, and a recommended route that includes sightseeing spots not set at the time of reservation as a waypoint. The service providing device 210 acquires operation information for selecting a route from the user device 300 (step S110).

Next, the service providing device 210 determines whether or not there is an area for automatic driving on the selected route (step S120). If there is no area for automatic driving (step S120: NO), the service providing device 210 causes the autonomous mobile body 100 to perform manual driving (step S122). The service providing device 210 determines whether a destination has been set during manual driving (step S124). If the destination is set (step S124: YES), the service providing device 210 returns the process to step S102, and if the destination is not set (step S124: NO), the service providing device 210 sets the preset prohibited area and The dynamic prohibited area is displayed superimposed on the video (step S126).

The user device 300 determines whether an operation to enter (approach) the preset prohibited area or the dynamic prohibited area has been received (step S128). If the user device 300 does not receive an operation to enter (approach) the preset prohibited area or the dynamic prohibited area (step S128: NO), the user device 300 continues manual operation (step S122). When the user device 300 receives an operation to enter (approach) the preset prohibited area or the dynamic prohibited area (step S128: YES), the user device 300 restricts manual operation by restricting the movement of the autonomous mobile object 100 (step S130). ). The user device 300 determines whether a dynamic prohibited area is detected on the road surface on which the autonomous mobile body 100 is traveling (step S132). When the user device 300 detects the dynamic prohibited area (step S132: YES), the user device 300 displays the dynamic prohibited area superimposed on the video (step S134). If the user device 300 does not detect a dynamic prohibited area (step S132: NO), the user device 300 continues manual operation (step S122).

If there is an area for automatic driving (step S120: YES), the service providing device 210 causes the autonomous mobile body 100 to perform automatic driving (step S140). The service providing device 210 determines whether the autonomous mobile body 100 has arrived at the destination (step S142). If the autonomous mobile body 100 has arrived at the destination (step S142: YES), the service providing device 210 advances the process to step S102. If the autonomous mobile body 100 has not arrived at the destination (step S142: NO), the service providing device 210 determines whether a dynamic prohibited area has been detected by the autonomous mobile body 100 (step S144). If the autonomous mobile body 100 does not detect a dynamic prohibited area (step S144: NO), the service providing device 210 continues automatic operation (step S140). When the autonomous mobile object 100 detects a dynamic prohibited area (step S144: YES), the service providing device 210 re-searches the route, displays the re-searched route (step S146), and continues automatic driving. (Step S140).

When terminating the remote tourism service (step S102: YES), the service providing device 210 searches for a route from the current position to the waiting area, and causes the autonomous mobile body 100 to automatically drive according to the searched route (step S150). ), the autonomous mobile body 100 is returned to the waiting area (step S152).

As described above, the remote tourism system 1 of the embodiment includes the prohibited area setting unit 214 or the detection unit 162 that sets a prohibited area where the autonomous mobile body 100 is prohibited from entering the tourist area, and the operating unit that receives user operations. 320, when the autonomous mobile body 100 is moving outside the prohibited area, the movement of the autonomous mobile body 100 is controlled based on the user's operation, and when the autonomous mobile body 100 receives an operation to enter the prohibited area, the autonomous mobile body 100 and a movement control section 160 that restricts movement of. The remote sightseeing system 1 of the embodiment includes the steps of setting a prohibited area in which the autonomous mobile body 100 is prohibited from entering the sightseeing area, accepting user operations, and moving the autonomous mobile body 100 outside the prohibited area. To realize a service providing method including the steps of: controlling the movement of the autonomous mobile body 100 based on the operation when the user enters the prohibited area; and restricting the movement of the autonomous mobile body 100 when the operation of entering a prohibited area is received. I can do it. According to the service providing device 210 of the embodiment, the prohibited area setting unit 214 sets a prohibited area in which the autonomous mobile body 100 is prohibited from entering a tourist area, the communication unit 211 receives user operation information, and the autonomous mobile When the body 100 is moving outside the prohibited area, the movement of the autonomous mobile body 100 is controlled based on the operation information, and when the operation information to enter the prohibited area is received, the movement of the autonomous mobile body 100 is restricted. . According to the autonomous mobile body 100 of the embodiment, the mobile body storage unit 170 stores a prohibited area in which the autonomous mobile body 100 is prohibited from entering a tourist area, the communication unit 110 receives user operation information, and the autonomous mobile body When the body 100 is moving outside the prohibited area, the movement of the autonomous mobile body 100 is controlled based on the operation information, and when an operation to enter the prohibited area is received, the movement of the autonomous mobile body 100 is restricted. A control unit 160 is provided. According to such an embodiment, even when the autonomous mobile body 100 is operated by a user, the autonomous mobile body 100 can be controlled in real time according to changes in the situation.

According to the remote tourism system 1 of the embodiment, the content providing unit 212 provides content in which the route information of the autonomous mobile body 100 and the prohibited area are superimposed on the video imaged by the autonomous mobile body 100. According to the remote tourism system 1 of the embodiment, when an operation to enter a prohibited area is received, it is possible to notify the user that the user's operations will be restricted. Thereby, according to the remote sightseeing system 1, it is possible to present to the user an area into which the autonomous mobile body 100 cannot enter and allow the user to manually operate the autonomous mobile body 100.

According to the remote tourism system 1 of the embodiment, a preset prohibited area (first prohibited area) in which the autonomous mobile object 100 cannot run in the sightseeing area is set as a prohibited area, and the preset prohibited area is entered. When the operation is accepted, the movement of the autonomous mobile body 100 can be stopped before entering the preset prohibited area, or the autonomous mobile body 100 can be moved to avoid the preset prohibited area. Thereby, according to the remote sightseeing system 1, it is possible to forcibly control the autonomous mobile body 100 from entering the preset prohibited area.

According to the remote tourism system 1 of the embodiment, a dynamic prohibited area (second prohibited area) detected by the autonomous mobile body 100 is set as a prohibited area, and when a dynamic prohibited area is detected, no movement is allowed. When an operation to enter a target prohibited area is received, the movement of the autonomous mobile body 100 can be stopped before entering the dynamic prohibited area, or the autonomous mobile body 100 can be moved to bypass the dynamic prohibited area. According to the remote tourism system 1 of the embodiment, if the distance to detour around the dynamic prohibited area does not exceed the threshold, the autonomous mobile body 100 is automatically detoured, and the distance to detour around the dynamic prohibited area exceeds the threshold. In this case, the autonomous mobile body 100 can be stopped. As a result, according to the remote sightseeing system 1, even if an obstacle occurs on the route traveled by the autonomous mobile body 100 due to a change in the weather, for example, the obstacle can be dynamically avoided.

According to the remote tourism system 1 of the embodiment, the detection unit 162 that recognizes an obstacle based on the image acquired by the autonomous mobile object 100 is provided, and the dynamic prohibited area is defined by the type of obstacle recognized by the detection unit 162. Can be set based on. According to the remote tourism system 1, for example, a process of recognizing a non-moving object such as a falling object and a moving object such as a pedestrian, detecting a dynamic prohibited area, and notifying the user of the dynamic prohibited area; Processing for controlling the autonomous mobile body 100 to avoid the dynamic prohibited area can be executed.

According to the remote tourism system 1 of the eighth embodiment, the detection unit 162 compares the image of the sightseeing area and the position information with the image and the position information acquired by the autonomous mobile object 100, and based on the comparison result, be able to recognize faults. As a result, the remote sightseeing system 1 can recognize a failure that did not exist before and execute processing to notify the user and processing to avoid the failure.

According to the remote tourism system 1 of the embodiment, the required time and distance to the destination are determined based on the prohibited area updated by the detection unit 162 or the prohibited area setting unit 214 as the route for the autonomous mobile body 100 to travel. It is possible to search for a route by searching for a second route that includes sightseeing spots other than the prioritized first route and the destination.

According to the remote tourism system 1 of the embodiment, if there is another autonomous mobile body closer to the destination than the autonomous mobile body 100 over which the user has authority, the user's authority cannot be transferred to the other autonomous mobile body. can. Thereby, according to the remote sightseeing system 1, the autonomous mobile body 100 can arrive at the destination in a short time.

According to the remote tourism system 1 of the embodiment, the user's authority can be transferred from the autonomous mobile body 100 for which the user has authority to another autonomous mobile body 100 that is permitted to travel in a specific area. As a result, according to the remote sightseeing system 1, for example, when manual driving or photographing of the autonomous mobile body 100 is restricted in a specific area, the authorized autonomous mobile body 100 can be used to sightsee the specific area. Further, according to the remote tourism system 1 of the embodiment, it is possible to charge a fee in order to obtain new authority for a specific area.

<Modified example>
In the embodiment, the autonomous mobile body 100 is controlled so as not to enter a prohibited area, but the present invention is not limited to this. For example, a preset prohibited area is not set in a place such as a store, and furthermore, an employee is not detected. However, there is no need to set a dynamic prohibited area. For example, the autonomous mobile object 100 approaches a store employee, and the remote tourism system 1 uses the autonomous mobile object 100, the service providing device 210, and the user device 300 to display product information at the store and to have a conversation with the store employee. may send and receive voice information, request payment for products based on user operations, request shipping of products, etc.

According to the remote tourism system 1 of the embodiment, a video taken of a sightseeing area and a position corresponding to the video are compared with a video taken by the autonomous mobile body 100 and a position acquired by the autonomous mobile body 100, The position acquired by the autonomous mobile body 100 can be corrected based on the comparison result. Thereby, according to the remote sightseeing system 1 of the embodiment, the position of the autonomous mobile body 100 can be recognized with high accuracy, and the autonomous mobile body 100 can be restricted from entering a prohibited area.

Although each embodiment and modification example have been described, this is only an example and is not limited to these. For example, each embodiment, each modification example, a part of each embodiment, or one of each modification example may be described. You may realize one aspect of the present invention by combining the section with one or more other embodiments or one or more other modifications.

DESCRIPTION OF SYMBOLS 1...Remote sightseeing system, 100...Autonomous mobile body, 110...Communication unit, 120...Camera unit, 130...Sensor unit, 140...Audio input/output unit, 150...Position acquisition unit, 160...Movement control unit, 162...Detection unit , 162a... Feature extraction unit, 162b... Recognition unit, 164... Position correction unit, 170... Mobile body storage unit, 180... Movement mechanism, 200... Tourist spot system, 210... Service providing device, 211... Communication unit, 212... Content Providing unit, 213... Obstacle recognition unit, 214... Prohibited area setting unit, 215... Route search unit, 216... Mobile object management unit, 217... Control unit, 220... Camera system, 230... Sensor system, 240... Store system, 250 ...Storage device, 300...User device, 302...Communication section, 304...Processing section, 306...Control section, 310...Sensor section, 320...Operation section, 330...Display section, 340...Audio input/output section, 400...Digital region Currency system, 500...Product payment system, 600...Authentication system, 700...Operator terminal

Claims (15)

a setting unit that sets a prohibited area in which autonomous mobile objects are prohibited from entering the tourist area;
an operation section that accepts user operations;
When the autonomous mobile body is moving outside the prohibited area set by the setting unit, the movement of the autonomous mobile body is controlled based on the operation received by the operation unit, and the movement of the autonomous mobile body is controlled based on the operation received by the operation unit. a control unit that restricts movement of the autonomous mobile body when receiving an operation to enter the prohibited area;
A remote tourism system equipped with The remote tourism system according to claim 1, further comprising a providing unit that provides content in which route information of the autonomous mobile body and the prohibited area set by the setting unit are superimposed on a video imaged by the autonomous mobile body. . The remote tourism system according to claim 2, wherein the providing unit notifies the user that the user's operations will be restricted when an operation to enter the prohibited area set by the setting unit is received. The prohibited area includes a first prohibited area in the tourist area in which the autonomous mobile object cannot run,
When the control unit receives an operation to enter the first prohibited area, the control unit stops the movement of the autonomous mobile body before entering the first prohibited area, or stops the autonomous mobile body from entering the first prohibited area. The remote sightseeing system according to any one of claims 1 to 3, wherein the remote sightseeing system is moved so as to avoid. The prohibited area includes a second prohibited area detected by the autonomous mobile body,
When the second prohibited area is detected and an operation to enter the second prohibited area is received, the control unit controls the autonomous mobile body before entering the second prohibited area. Stopping the movement or moving around the second prohibited area;
A remote tourism system according to any one of claims 1 to 4. The control unit automatically detours the autonomous mobile body when the distance to detour around the second prohibited area does not exceed a threshold, and when the distance to detour around the second prohibited area exceeds the threshold. The remote sightseeing system according to claim 5, wherein the autonomous mobile body is stopped. comprising a detection unit that recognizes a failure based on the image acquired by the autonomous mobile body,
The second prohibited area is set based on the type of obstacle recognized by the detection unit.
The remote tourism system according to claim 5. 8. The detection unit compares an image captured of the sightseeing area and position information with an image and position information acquired by the autonomous mobile object, and recognizes the obstacle based on the comparison result. remote tourism system. comprising a route search unit that searches for a first route that prioritizes the time and distance to the destination and a second route that includes tourist spots other than the destination as a route for the autonomous mobile body to travel;
The setting unit updates the prohibited area based on the obstacle detected by the autonomous mobile body,
The route search unit searches for the route based on the prohibited area updated by the setting unit.
A remote tourism system according to claim 7 or 8. 10. The method according to claim 1, further comprising a management unit that transfers the user's authority to the other autonomous mobile body when there is another autonomous mobile body closer to the destination than the autonomous mobile body for which the user has authority. The remote tourism system described in any one of these items. Any one of claims 1 to 9, further comprising a management unit that transfers the user's authority from the autonomous mobile body over which the user has authority to another autonomous mobile body that is permitted to run in a specific area. Remote tourism system as described in section. The autonomous mobile body compares a video taken of the tourist area and a position corresponding to the video with a video taken by the autonomous mobile body and a position acquired by the autonomous mobile body, and based on the comparison result, The remote sightseeing system according to any one of claims 1 to 11, further comprising a position correction unit that corrects a position acquired by the autonomous mobile body. a setting unit that sets a prohibited area in which autonomous mobile objects are prohibited from entering the tourist area;
a communication unit that receives user operation information;
When the autonomous mobile body is moving outside the prohibited area set by the setting unit, the movement of the autonomous mobile body is controlled based on the operation information, and the movement of the autonomous mobile body is controlled in the prohibited area set by the setting unit. a control unit that limits movement of the autonomous mobile body when receiving operation information to enter the autonomous mobile body;
A service providing device comprising: a storage unit that stores a prohibited area in which autonomous mobile objects are prohibited from entering a tourist area;
a communication unit that receives user operation information;
When the autonomous mobile body is moving outside the prohibited area stored in the storage unit, the movement of the autonomous mobile body is controlled based on the operation information, and the movement of the autonomous mobile body is controlled in the prohibited area stored in the storage unit. a control unit that limits movement of the autonomous mobile body when an operation to enter the autonomous mobile body is received;
An autonomous mobile body equipped with setting a prohibited area in which autonomous mobile objects are prohibited from entering the tourist area;
a step of accepting user operations;
When the autonomous mobile body is moving outside the prohibited area, the movement of the autonomous mobile body is controlled based on the operation, and when the operation to enter the prohibited area is received, the movement of the autonomous mobile body is controlled. a step to limit the
service provision methods, including

Images