Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
  • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00

Definitions

• This disclosure relates to an information processing device, an information processing method, and a program.
• Patent Document 1 describes the automatic generation of a movement path for a moving object such as a robot from an input image. Specifically, in Patent Document 1, free space areas and obstacle areas are identified from the image, and the movement path is searched for by calculating the cost of movement.
• Patent Document 1 searches for a travel route while taking into account only obstacles, and does not take into account other conditions. As a result, it is unclear whether the route searched for is appropriate, and it is difficult to search for an appropriate route.
• the purpose of this disclosure is therefore to solve the problem mentioned above, that is, the difficulty of searching for a more appropriate route from an image.
• An information processing device includes: a calculation unit that calculates, for each divided image obtained by dividing an image into a plurality of regions, situation information that indicates a situation of a road surface reflected in the divided image based on the divided image; an evaluation unit that evaluates a route formed on the image by connecting a predetermined set of the divided images based on the situation information of a road surface reflected in the divided images included in the route; Equipped with The structure is as follows.
• an information processing method includes: calculating situation information representing a situation of a road surface reflected in each divided image obtained by dividing the image into a plurality of regions based on the divided image; evaluating a route formed on the image by connecting a predetermined set of the divided images based on the situation information of a road surface shown in the divided images included in the route;
• the structure is as follows.
• a program includes: calculating situation information representing a situation of a road surface reflected in each divided image obtained by dividing the image into a plurality of regions based on the divided image; evaluating a route formed on the image by connecting a predetermined set of the divided images based on the situation information of a road surface reflected in the divided images included in the route; Have a computer carry out the process,
• the structure is as follows.
• FIG. 1 is a diagram showing a usage situation of the present disclosure.
• 1 is a block diagram showing a configuration of a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 5 is a flowchart showing a processing operation of the first information processing device according to the present disclosure.
• FIG. 11 is a block diagram showing a configuration of a third information processing device according to the present disclosure.
• FIG. 11 is a diagram showing a process performed by a third information processing device according to the present disclosure.
• FIG. 13 is a block diagram showing a hardware configuration of a fifth path evaluation device according to the present disclosure.
• FIG. 13 is a block diagram showing a configuration of a fifth path evaluation device according to the present disclosure.
• the information processing device 10 in this embodiment functions as a route evaluation device that evaluates a route along which a moving object moves from an image.
• a moving object M is a robot that carries a baggage B in a predetermined place P such as a warehouse, and as shown in FIG. 1, when the moving object M moves from its current position to a destination E, the information processing device 10 acquires an image of the place P in the warehouse photographed by the photographing device C, and evaluates a route from the image to the destination E.
• the moving object M is not limited to being a robot that carries a baggage B, and may be a person or a vehicle operated or boarded by a person.
• the information processing device 10 can be applied to evaluating a route along which any moving object may move, and can also be applied to a route related to movement for any purpose.
• the information processing device 10 is composed of one or more information processing devices each having a calculation device and a storage device. As shown in FIG. 2, the information processing device 10 is composed of a traffic area detection unit 11, a road surface condition calculation unit 12, a route evaluation unit 13, an output unit 14, and a control unit 15. The functions of the traffic area detection unit 11, the road surface condition calculation unit 12, the route evaluation unit 13, the output unit 14, and the control unit 15 can be realized by the calculation device executing a program for realizing each function stored in the storage device.
• the information processing device 10 is also composed of an image storage unit 16 and a route information storage unit 17. The image storage unit 16 and the route information storage unit 17 are composed of storage devices. Each component will be described in detail below.
• the image storage unit 16 acquires an image of a predetermined location P within the warehouse where the moving object M may move, captured by the imaging device C, and stores the image in the image storage unit 16.
• the imaging device C is, for example, a stereo camera (RGB sensor), an RGB-D sensor, or a LiDAR (Light Detection and Ranging) that can capture an image including distance information to the target object.
• the imaging device C is installed above the predetermined location P and is set up to capture an image of the predetermined location P viewed from above. Therefore, the image shows the road surface on which the legs, such as the wheels, of the moving object M touch the ground.
• multiple image capture devices C may be installed.
• multiple image capture devices C of the same type that use the same method for acquiring images may be installed, or multiple types of image capture devices C may each be installed in one or more units.
• the image capture device C may be installed in any location, for example, on a moving object M.
• the image capture device C may be attached to a handle on the dolly facing slightly downward, allowing the road surface to be captured in the same manner as described above.
• only one image capture device C may be installed on the moving object M, and the road surface at the above-mentioned specified location P may be sequentially converted into a three-dimensional image (point cloud) from images captured at multiple times by using a technique such as Structure from Motion (SfM).
• SfM Structure from Motion
• the images stored in the image storage unit 16 are updated with new images captured by the imaging device C at regular time intervals or at any timing. This allows the images to become images that represent the latest or most recent state at a given location P. Note that the images stored in the image storage unit 16 and that are to be processed as described below are not necessarily limited to images that include distance information, and may be any images.
• the passage area detection unit 11 reads out the image G stored in the image storage unit 16 as shown in FIG. 3 (3-1), and uses the image G to detect a passage area in which a moving body M can pass at a location P corresponding to the image G. Specifically, the passage area detection unit 11 detects an area that forms a substantially identical plane in the image, and detects the area as a passage area. As an example, the passage area detection unit 11 extracts a subset of the point cloud captured in the captured image as individual objects, and extracts objects such as horizontal and vertical planes based on distance information in the image. Then, the passage area detection unit 11 detects a horizontal plane having a predetermined width from the image, excluding objects that form a step portion including a vertical plane, as a passage area.
• the passage area detection unit 11 detects a passage area Q as shown by the hatching in FIG. 3 (3-2) from the image shown in FIG. 3 (3-1).
• the passage area detection unit 11 may simply detect the widest area of approximately the same horizontal plane as the passage area based on the distance image included in the image, or may detect it using other methods. Also, the passage area detection unit 11 does not need to be provided, and the following processing may be performed with the entire image as the passage area.
• the road surface condition calculation unit 12 calculates condition information that represents the condition of the road surface in the travel area of image G.
• undulation condition information that represents the undulation condition of the road surface that is, the amount of undulation of the road surface
• the road surface condition calculation unit 12 divides image G into a plurality of regions, and calculates the amount of undulation of the road surface for each divided image g that is each divided region. Note that while image G is displayed in FIG. 4, a moving object M located at a predetermined location P and a destination E captured in image G are illustrated in correspondence with the image G.
• the road surface condition calculation unit 12 calculates the amount of undulation based on the above-mentioned passage area surface based on the distance information included in the divided image g. Specifically, the road surface condition calculation unit 12 calculates the difference between the height of the passage area surface and each of the highest and lowest points in the divided image g, assuming the height of the passage area surface to be 0, and calculates the greater absolute value of the difference as the amount of undulation x_i of each divided image g(i). Therefore, if a point higher than the passage area surface is included, the amount of undulation is positive, and if a point lower than the passage area surface is included, the amount of undulation is negative, which can be recognized as a convex obstacle or a concave obstacle, respectively.
• the difference in height will be large, and the absolute value of the amount of undulation will be calculated to be large.
• the absolute value of the amount of undulation will be calculated to be small. Therefore, in this embodiment, the greater the actual amount of undulation, the greater the absolute value of the amount of undulation.
• the value of the amount of undulation itself is expressed as a positive or negative value, but when referring to the magnitude of the amount of undulation, it represents the magnitude of the undulation, that is, the height or depth relative to the surface of the passage area, and is expressed as a value with an absolute value added to the amount of undulation.
• the road surface condition calculation unit 12 may predefine a reference height difference diff_base at which the amount of undulation is 1.0, and may divide the height difference diff calculated for each divided image g by the reference height difference diff_base (normalized value) as the amount of undulation x_i for each divided image g(i). The road surface condition calculation unit 12 then stores the amount of undulation calculated for each divided image g in the image storage unit 16 in association with the corresponding divided image g.
• the road surface condition calculation unit 12 may calculate the above-mentioned amount of undulation using information other than the distance information included in the above-mentioned divided image g.
• the road surface condition calculation unit 12 may acquire topographical information of a predetermined location P in the warehouse corresponding to image G consisting of the above-mentioned divided images g, and calculate the amount of undulation using such topographical information and the distance information included in the divided image g.
• the topographical information of the predetermined location is assumed to be a three-dimensional model including information on the surrounding environment such as the placement of objects and the positions of walls and pillars.
• objects for which distance information could not be measured in image G can be extracted from the topographical image and associated with the divided image g, and the amount of undulation in the divided image g including such objects can be calculated.
• FIG. 5 shows the state when the above-mentioned amount of undulation is calculated for each divided image g.
• FIG. 5 illustrates multiple divided area images a corresponding to multiple divided images g, and area image A, which is the entire divided area image a, can correspond to image G.
• Each divided area image a is filled with a specified color, and the amount of undulation is represented by this color, i.e., the shading value.
• each divided area image a is displayed with a shading value corresponding to the amount of undulation calculated for the corresponding divided image g.
• the darker the shading value the greater the amount of undulation, and the majority of image G has a small amount of undulation, but there are areas with a large amount of undulation in places.
• FIG. 5 shows area image A corresponding to image G, and a moving object M and a destination E located at a specified place P photographed in image G are illustrated corresponding to area image A.
• the road surface condition calculation unit 12 may also calculate the undulating direction of the road surface as the above-mentioned undulating condition of the road surface. Specifically, the road surface condition calculation unit 12 calculates the inclination direction from the low point to the high point in the divided image g as the undulating direction for each divided image g based on the distance information included in the divided image g. Then, the road surface condition calculation unit 12 stores the undulating direction calculated for each divided image g in the image storage unit 16 in association with the corresponding divided image g. Note that the road surface condition calculation unit 12 may calculate the above-mentioned undulating direction using information other than the distance information included in the divided image g.
• the road surface condition calculation unit 12 may calculate the undulating direction using the topographical information of the above-mentioned predetermined location P and the distance information included in the divided image g.
• the road surface condition calculation unit 12 may calculate the undulating direction using the topographical information of the above-mentioned predetermined location P and the distance information included in the divided image g.
• the road surface condition calculation unit 12 may calculate the road surface conditions, such as the amount of undulation and the direction of undulation, from the divided image g using a model previously constructed by machine learning.
• the model is generated by machine learning learning data in which the divided image g is associated with the amount of undulation and the direction of undulation. Then, by inputting the divided image g into the model, the road surface condition calculation unit 12 can obtain the amount of undulation and the direction of undulation corresponding to the divided image g as an output.
• the image G constituting the divided image g does not necessarily need to include distance information.
• the road surface condition calculation unit 12 is not limited to calculating information representing the road surface condition, such as the amount and direction of undulation of the road surface, using the method described above, and may perform the calculations using any method.
• the road surface condition calculation unit 12 may calculate the normal of each point on the road surface from the distance information included in the divided image g, and calculate the amount and direction of undulation from the variance of the normal.
• the route evaluation unit 13 (evaluation unit) generates one or more routes on the image G along which the moving body M may move, and evaluates each route by calculating an evaluation value. At this time, the route evaluation unit 13 calculates the evaluation value of the route based on the amount of undulation in the divided image g included in the route. As an example, the route evaluation unit 13 calculates a "score", which is an evaluation value representing the ease of movement of the moving body M along the route, using the following formula 1.
• the magnitude of the amount of undulation i.e., the absolute value of the amount of undulation, is a value equal to or greater than 0, and the greater the actual undulation, the larger the value.
• the evaluation value score is set to a value of 0 ⁇ score ⁇ 1, and according to the above formula 1, the greater the average undulation of the route, the smaller the value (closer to 0), indicating that travel is more difficult, and the smaller the average undulation of the route, the larger the value (closer to 1), indicating that travel is easier.
• the route evaluation unit 13 associates the calculated evaluation value with the position information of the divided images g located on the route for each route, and stores them in the route information storage unit 17.
• the route evaluation unit 13 may calculate the route evaluation value using a method different from that described above. For example, the route evaluation unit 13 may calculate the evaluation value based on the maximum amount of undulation on the route. In this case, if the maximum amount of undulation on the route exceeds a threshold value set according to the moving body M, the evaluation value may be calculated to be smaller or may be calculated to be unassessable. This is because the amount of undulation may be an amount that the moving body M cannot overcome in the first place, or may be an amount of undulation that may cause inconvenience such as the possibility of baggage B falling. Taking such things into consideration, the evaluation value may be calculated to be smaller or may be calculated to be unassessable as described above.
• the route evaluation unit 13 may also calculate the evaluation value of the route based on the undulation direction in the divided image g included in the route, in addition to the amount of undulation described above. For example, when a divided image g in which the undulation increases in the travel direction of the route is present, the route evaluation unit 13 calculates the evaluation value score of the route to be lower. On the other hand, when a divided area g in which the undulation decreases in the travel direction of the route is present, the route evaluation unit 13 calculates the evaluation value score of the route to be higher. Note that the route evaluation unit 13 may calculate the evaluation value score of the route based only on the undulation direction in the divided image g included in the route.
• the route evaluation unit 13 selects the shortest route and settable routes as candidates for routes that the moving body M can travel on the image G, and calculates an evaluation value for each route. At this time, the route evaluation unit 13 may find the route with the best evaluation value, or may find several routes with the top evaluation values and store them in the route information storage unit 17. At this time, the route with the best evaluation value can be found using, for example, the Dijkstra algorithm.
• the road surface condition calculation unit 12 may calculate the undulation condition such as the amount of undulation and the direction of undulation of the road surface for a new divided image g' obtained by changing the size of the divided area of the divided image g in the image G in the same manner as described above.
• FIG. 6 (6-1) shows a divided image g corresponding to a part of image G
• FIG. 6 (6-2) shows a new divided image g' obtained by changing the size of the divided image g.
• FIG. 6 (6-1') shows a shading value corresponding to the amount of undulation in each divided area image a of the area image A corresponding to FIG. 6 (6-1), and FIG.
• FIG. 6 (6-2') shows a shading value corresponding to the amount of undulation in each divided area image a' of the area image A corresponding to FIG. 6 (6-2).
• the divided image g in FIG. 6 (6-1) a view of the road surface corresponding to the divided image g from the side is shown above, and there are two deep cracks in the road surface corresponding to the divided image g, as indicated by the symbol f.
• the divided area image a located between the two cracks f is calculated to have a small amount of undulation, but as shown in FIG. 6 (6-2'), in the new divided area image a' corresponding to the new divided area image g' in which the divided area is expanded, the two cracks f are located within one divided area image a', and the amount of undulation can be calculated to be large.
• the route evaluation unit 13 calculates an evaluation value of the route formed on the new divided image g'. That is, for a route formed by connecting the new divided images g', the route evaluation unit 13 calculates an evaluation value from the undulating conditions, such as the amount of undulation and the direction of undulation, calculated from the new divided image g'. In this way, when the size of the divided images g, g' is changed, evaluation values of the routes formed before and after the change can be obtained.
• the divided images g, g' may be changed to various sizes as described above, and various routes in each case may be evaluated.
• the output unit 14 displays the information obtained by the above-mentioned process on the information processing terminal of a user such as an administrator who manages the movement of the moving object M. For example, the output unit 14 displays the route for which the evaluation value has been calculated, superimposed on the area image A corresponding to the image G, as shown in FIG. 7. That is, the output unit 14 displays the route as a line superimposed on each divided area image a corresponding to each divided image g included in the route. As an example, in FIG. 7, the output unit 14 displays the route R1, which has the best evaluation value, that is, which has been evaluated as the easiest route to travel, as a solid line, and displays the route R2, which has a lower evaluation value but is the shortest distance, as a dotted line.
• the output unit 14 may display the top several routes, or may display routes selected based on other criteria.
• the output unit 14 may also display information based on the evaluation result together with the route.
• the output unit 14 may display the evaluation value score together with the route, as shown in FIG. 7, or may display the evaluation ranking or information explaining the characteristics of the route.
• the output unit 14 may display the amount of undulation calculated for the corresponding divided image g on the divided area image a in addition to displaying the above-mentioned route.
• the output unit 14 displays the divided area image a portion by filling it with a color of a shading value corresponding to the amount of undulation. In this case, it is assumed that the darker the shading value, the greater the amount of undulation.
• the output unit 14 calculates the direction of undulation for each divided image g as described above, it may display the direction of undulation on the divided area image a corresponding to the divided image g so that the direction of undulation can be seen.
• FIG. 1 an example of the divided area image a is shown in FIG.
• the amount of undulation may be displayed on the divided area image a with a shading value, and the direction of undulation may be displayed with an arrow. In this case, it is indicated that the undulation becomes higher in the direction of the arrow.
• the output unit 14 may display the direction of undulation with a shading value on the divided area image a. For example, the shading value is displayed so that it becomes darker in the direction from the low undulation to the high undulation.
• Figures 8 (8-2) to (8-4) each show an example in which the undulation direction is displayed using a gray value, with the lower diagram showing a schematic representation of the road surface condition.
• Figure 8 (8-2) shows a case in which the amount of undulation increases from right to left
• Figure 8 (8-3) shows a case in which the road surface is convex in the center
• Figure 8 (8-4) shows a case in which the road surface is concave in the center.
• the output unit 14 may display the undulation direction in any display format.
• the output unit 14 may display other road surface conditions that are different from the amount of undulation and direction described above.
• the output unit 14 may display the area image A consisting of the divided area image a as described above as an image G consisting of the divided image g.
• the output unit 14 may display to the user an image G of a specific location P in the warehouse that was actually photographed, and may superimpose the route, amount of undulation, direction of undulation, etc. on the image G.
• the control unit 15 outputs a control command to control the movement of the moving body M so that the moving body M moves along the route evaluated as described above within a specified location P in the warehouse corresponding to the image G.
• the control unit 15 controls the moving body M to move along the route R1 with the best evaluation value, or controls the moving body M to move along the selected route in response to a route selection by the user who displayed the route.
• the moving body M moves to the destination E in accordance with the control command.
• the control unit 15 does not necessarily need to be provided, and the moving body M may be operated by a user, etc.
• the information processing device 10 detects a passage area from an image G of a predetermined place P in a warehouse where a moving object M can move, which is photographed by the photographing device C (step S1 in FIG. 9). For example, the information processing device 10 detects, as the passage area Q, a horizontal plane shown in the shaded area in FIG. 3 (3-2) obtained by extracting and removing steps and the like from the image G as shown in FIG. 3 (3-1).
• the information processing device 10 divides the image G into a plurality of regions as shown in FIG. 4 (step S2 in FIG. 9), and calculates the amount of undulation of the road surface shown in the image for each divided region, that is, divided image g (step S3 in FIG. 9). For example, based on the distance information included in the divided image g, the information processing device 10 calculates the amount of undulation as the greater absolute value of the difference between the highest point and the lowest point in the divided image g relative to the height of the reference passage area surface, as described above. Alternatively, the information processing device 10 may calculate the amount of undulation of the road surface from the divided image g using a model previously constructed by machine learning.
• the information processing device 10 may calculate, for each divided image g, the undulation direction, which is the inclination direction from the low point to the high point in the divided image g, based on the distance information contained in the divided image g, and may further calculate other road surface conditions.
• the information processing device 10 searches for one or more routes on the image G along which the moving body M can move, and calculates and evaluates each route (step S4 in FIG. 9). At this time, the information processing device 10 evaluates the route so that the evaluation value score representing the ease of movement of the route is smaller as the amount of undulation in the divided image g included in the route is larger, and the evaluation value score is larger as the amount of undulation in the divided image g included in the route is smaller. The information processing device 10 may also calculate the evaluation value of the route based on the undulation direction in the divided image g included in the route.
• the information processing device 10 may calculate the undulation conditions such as the amount of undulation and the direction of undulation of the road surface for a new divided image g' obtained by changing the size of the divided area of the above-mentioned divided image g in the image G in the same manner as described above, and calculate the evaluation value of the route formed on the new divided image g' (steps S3 and S4 in FIG. 9).
• the information processing device 10 outputs to the information processing terminal of a user, such as an administrator who manages the movement of the mobile unit M, to display the evaluated route (step S5 in FIG. 9). For example, as shown in FIG. 7, the information processing device 10 displays the routes R1 and R2 for which the evaluation values have been calculated, superimposed on the area image A corresponding to the image G. At this time, the information processing device 10 may display other routes (e.g., the route R2 with the shortest distance) without being limited to the route R1 with the best evaluation value, and may also display the evaluation value score. Furthermore, the information processing device 10 may display the amount of undulation and the direction of undulation on the divided area image a corresponding to the divided area image a in addition to displaying the route. For example, as shown in FIG. 7, the divided area image a is displayed in a color with a shading value corresponding to the amount of undulation, and the direction of undulation is displayed with an arrow or shading value.
• the divided area image a is displayed in a color with
• the information processing device 10 controls the movement operation of the moving body M so that the moving body M moves along the evaluated route within a predetermined location P in the warehouse corresponding to the image G (step S6 in FIG. 9). Note that the movement of the moving body M may be controlled by a user or the like.
• the information processing device 10 in this embodiment calculates the amount and direction of undulation of the road surface shown in the image, and evaluates the route according to this undulation. In other words, it evaluates the ease of movement according to the undulations present on the route. Therefore, it is possible to search for a route suitable for movement according to a moving body M such as a robot.
• the information processing device 10 in this embodiment has a configuration similar to that of the information processing device described in the above-mentioned embodiment.
• the information processing device 10 in this embodiment has a configuration that evaluates a route by taking into account the road surface conditions, such as the amount of undulation and the direction of undulation, as well as the road surface conditions, such as the slipperiness, which indicates the slipperiness of the road surface.
• the road surface conditions such as the amount of undulation and the direction of undulation
• the road surface conditions such as the slipperiness
• the road surface condition calculation unit 12 uses the divided images g of the image G in the same manner as described above to calculate the sliding property information representing the slipperiness of the road surface for each divided image g.
• the road surface condition calculation unit 12 extracts the feature amount of the divided image g, that is, the feature amount due to the texture of the road surface reflected in the divided image g, and calculates the sliding property value representing the slipperiness of the road surface from the feature amount.
• the road surface condition calculation unit 12 uses a model previously constructed by machine learning, and obtains the sliding property value of the road surface output by inputting the divided image g to the model.
• the model is generated by machine learning learning data in which the divided image g and the sliding property value are associated.
• the sliding property value representing the slipperiness is calculated to be high
• the sliding property value representing the slipperiness is calculated to be low
• the sliding property value representing the slipperiness is calculated to be high
• the value of the frictional property is set, for example, between 0 and 1, with a higher value indicating greater slipperiness.
• the road surface condition calculation unit 12 is not limited to calculating the frictional property of the road surface using the method described above, and may calculate the frictional property using any method.
• the route evaluation unit 13 calculates an evaluation value "score" of the route by taking into consideration the amount of undulation and the direction of undulation as well as the sliding property calculated as described above.
• the route evaluation unit 13 calculates "score", which is an evaluation value representing the ease of movement of the moving body M along the route, using the following formula 2. Note that, although formula 2 uses only the amount of undulation as the undulating condition of the road surface, the evaluation value "score" may also be calculated by using the direction of undulation as described above.
• the route evaluation unit 13 associates the calculated evaluation value with the position information of the divided images g located on the route for each route, and stores them in the route information storage unit 17.
• the route evaluation unit 13 may calculate the route evaluation value using a method different from that described above. For example, the route evaluation unit 13 may calculate the evaluation value based on the maximum sliding property on the route. In this case, if the maximum sliding property on the route exceeds a threshold value set according to the moving body M, the evaluation value may be calculated to be smaller or may be calculated to be unassessable. This may be the case when the sliding property is such that the moving body M will slip and will not be able to move, and taking such a case into consideration, the evaluation value may be calculated to be smaller or may be calculated to be unassessable as described above.
• the road surface condition calculation unit 12 and the route evaluation unit 13 may also calculate the frictional properties of the road surface for a new divided image g' obtained by changing the size of the divided area of the divided image g in the image G in the same manner as described above, and use the frictional properties to calculate the route evaluation value.
• the output unit 14 displays the evaluated route and displays the evaluation value together with the route, similar to the first embodiment.
• the control unit 15 may output a control command to control the movement operation of the moving object M so that the moving object M moves according to the evaluated route within a predetermined location P in the warehouse corresponding to the image G, similar to the first embodiment.
• the information processing device 10 in this embodiment calculates the slipperiness, which represents the slipperiness of the road surface, in addition to the amount and direction of undulation of the road surface shown in the image, and evaluates the route according to these conditions. In other words, the ease of movement is evaluated according to the undulations and slipperiness conditions present on the route. Therefore, it is possible to search for a route suitable for movement according to a moving body M such as a robot.
• the information processing device 10 in this embodiment has a configuration similar to that of the information processing device described in the above-mentioned embodiment, and in addition has a mobile object information storage unit 18 configured with a storage device.
• the information processing device 10 in this embodiment has a function of calculating an evaluation value of a route according to the characteristics of the mobile object M. Below, the configuration that differs from the other embodiments will be described in detail.
• the moving body M is, for example, a person, a cart or stroller operated by a person, a wheelchair or smart mobility that a person rides on, multiple automated guided vehicles with different wheel diameters, etc.
• the information processing device 10 acquires moving body information that indicates the characteristics of the moving body M that can move along the route, and stores it in the moving body information storage unit 18.
• the moving body information is input to the information processing device 10 from the terminal of a user who wishes to search for a route.
• a user himself wishes to search for a route when heading to a destination, or when a user who manages the transportation of luggage in a warehouse wishes to search for a route to transport luggage using a specific automated guided vehicle.
• the moving body information is information that represents the characteristics of the moving body M, and includes the type of moving body.
• the type of moving body is information that represents the moving method by the moving body, and includes walking (bipedal walking), cart/stroller (bipedal walking and wheel movement), wheelchair/smart mobility (wheel movement), and automated guided vehicle (wheel movement).
• the moving body information may also include information that represents the characteristics of each type of moving body. For example, if the moving body is a person and the moving method is walking, the moving body information may include information that represents the physical characteristics of the person, and for example, information such as the person's gender, age, and medical history.
• the moving body information may include information about the wheels of the moving body, and for example, information about the size (diameter and thickness) of the wheels. If the moving method is wheel movement, it may also include information about the presence or absence of a transported object and its weight.
• the above-mentioned moving body information is only an example, and is not limited to the above-mentioned information, and may include any information that represents the characteristics of the moving body.
• the route evaluation unit 13 uses the undulations and sliding properties of the divided images g on the route as described in other embodiments, and also calculates the evaluation value of each route based on the above-mentioned moving body information. For example, if the moving method included in the moving body information is walking (bipedal walking), the divided images g included in the route are thinned out, and the evaluation value of the route is calculated using the undulations of the remaining divided images g. This is because when a person walks on two legs, the foot touches the ground at a predetermined distance depending on the stride, and the person may straddle a number of divided images g corresponding to the stride.
• the route evaluation unit 13 thins out every other divided image g on the route, calculates the average undulation of all the remaining divided images g on the route, and calculates the evaluation value score from this average undulation.
• the amount of thinning out of the divided images g on the route is set according to the physical characteristics of the person included in the moving body information.
• the information processing device 10 stores stride information corresponding to the physical characteristics of a person in advance, sets the stride based on the gender and age included in the moving body information, and sets the thinning amount corresponding to the stride. Therefore, as an example, the thinning amount is set smaller for women than for men, the older the person, the smaller the thinning amount is set, and further, the smaller the thinning amount is set if the medical history is of a specific type.
• the route evaluation unit 13 may change the method of calculating the route evaluation value according to the characteristics included in the mobile body information. For example, if the mobile body information includes a specific medical history (for example, a knee injury, a cardiopulmonary disease, etc.) in the physical characteristics of a person, the evaluation value score may be multiplied by a coefficient ⁇ (0 ⁇ 1) corresponding to the specific medical history, or the undulation amount of each divided image g may be multiplied by a coefficient ⁇ (1 ⁇ ) corresponding to the specific medical history. In this way, the evaluation value of the route is calculated to be low according to the characteristics such as the person's medical history, and the route is evaluated taking into account the risks during movement.
• a specific medical history for example, a knee injury, a cardiopulmonary disease, etc.
• the evaluation value and the undulation amount may be multiplied by a coefficient (weight) according to the size, the presence or absence of a transported object, and the weight. For example, the smaller the wheels are, the greater the weight is applied to the undulation amount, and the evaluation value may be calculated by applying a greater weight to the undulation amount when a transported object is present or the weight is large.
• the above-mentioned method of calculating the evaluation value by the route evaluation unit 13 is just one example, and the evaluation value may be calculated using another calculation method according to the mobile object information.
• the route evaluation unit 13 may calculate an evaluation value by taking into account the moving body information in the same manner as described above, for a route set on a new divided image g' whose size has been changed.
• the output unit 14 displays the evaluated route, as in the first embodiment. For example, if the user requests a route to the destination by walking, the route for which the evaluation value has been calculated is superimposed and displayed on the terminal T of the user U on the area image A corresponding to the image G, as shown in FIG. 11. As an example, in FIG. 11, the output unit 14 displays the route R1, which has the best evaluation value, i.e., the route evaluated as being the easiest to travel, as a solid line, and displays the route R2, which has a lower evaluation value but is the shortest distance, as a dotted line, and further displays the evaluation value score together with the route.
• the route R1 which has the best evaluation value, i.e., the route evaluated as being the easiest to travel
• the route R2 which has a lower evaluation value but is the shortest distance, as a dotted line
• the information processing device 10 in this embodiment evaluates a route according to the characteristics of the moving object, in addition to the undulating condition of the road surface shown in the image, such as the amount and direction of undulation. Therefore, it is possible to search for a route suitable for movement according to a moving object such as a person or a robot.
• the information processing device 10 in this embodiment is configured to be used in the healthcare field to provide a menu optimized for a user.
• a user requests a search for a movement route for exercise, such as a jogging route or a walking route, will be described.
• the information processing device 10 acquires user information (personal information) representing the characteristics of the user from the user who is a moving body.
• user information information representing exercise-related characteristics such as the user's gender, age, health condition, and exercise preferences is acquired.
• the health condition is information representing the user's health condition, such as sleep time and fatigue level
• the exercise preferences are information such as the exercise content desired by the user, such as exercise intensity (e.g., hard running, gentle jogging, walking, etc.), exercise distance and time, and exercise goals (e.g., hill training, flat places, avoid crowded places, etc.).
• the information processing device 10 acquires information on the user's current location and the exercise start location, and acquires images of locations where a route can be set according to the location information.
• the information processing device 10 acquires and stores images such as database images, satellite images, and security camera images provided on the Internet.
• the information processing device 10 calculates conditions such as the amount of undulation of the road surface from the image, as described above, and calculates an evaluation value of a route that can be a candidate for exercise such as jogging by the user.
• the evaluation value of the route represents the ease of movement, but in this embodiment, since the route is one that involves movement such as walking or running, a value representing the exercise load is calculated from the evaluation value described above. For this reason, in this embodiment, the higher the ease of movement, which is the evaluation value described above, the smaller the exercise load value is calculated, and the lower the ease of movement, the larger the exercise load value is calculated.
• the greater the amount of undulation on the route the greater the exercise load of the route is calculated, and the smaller the amount of undulation on the road, the smaller the exercise load of the route is calculated.
• the information processing device 10 sets multiple routes and calculates the exercise load for each route.
• the information processing device 10 selects a route optimized for the user based on user information that represents the user's exercise characteristics. For example, if the user desires hard running as the exercise intensity or desires hill training as the exercise goal, a travel route with a high exercise load is selected. On the other hand, if the user desires a flat place as the exercise goal, a travel route with a low exercise load is selected.
• the information processing device 10 may also select a travel route taking into account the user's characteristics such as health condition and age. For example, if the user's health condition is high in fatigue or the user is old, a travel route with a low exercise load may be selected without forcing the user.
• the information processing device 10 outputs the route optimized for the user according to the user information as described above to be displayed on the user's information processing terminal. This allows the user to obtain exercise information that is a walking or running route that is suitable for the user.
• the information processing device 10 may also output information on the exercise load of each of the multiple routes to be displayed on the user's information processing terminal. The user who receives this can select his or her own travel route by referring to the multiple routes and the exercise load values. In this way, the information processing device 10 in this embodiment can support the user's decision-making.
• the path evaluation device 100 is configured as a general information processing device, and is equipped with the following hardware configuration, as an example.
• ⁇ CPU Central Processing Unit
• ROM Read Only Memory
• RAM Random Access Memory
• Program group 104 loaded into RAM 103
• a storage device 105 for storing the program group 104
• a drive device 106 that reads and writes data from and to a storage medium 110 outside the information processing device.
• a communication interface 107 that connects to a communication network 111 outside the information processing device
• Input/output interface 108 for inputting and outputting data
• a bus 109 that connects each component
• FIG. 12 shows an example of the hardware configuration of the information processing device that is the path evaluation device 100, and the hardware configuration of the information processing device is not limited to the above-mentioned case.
• the information processing device may be configured with a part of the above-mentioned configuration, such as not having the drive device 106.
• the information processing device may use a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these.
• the path evaluation device 100 can be equipped with the calculation unit 121 and evaluation unit 122 shown in FIG. 13 by having the CPU 101 acquire and execute the program group 104.
• the program group 104 is stored in advance in the storage device 105 or ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as necessary.
• the program group 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read out the programs and supply them to the CPU 101.
• the calculation unit 121 and evaluation unit 122 described above may be constructed of dedicated electronic circuits for realizing such means.
• the calculation unit 121 calculates, for each divided image obtained by dividing an image into a plurality of regions, situation information that indicates the situation of the road surface reflected in the divided image based on the divided image.
• the evaluation unit 122 evaluates a route that is formed on the image by connecting a predetermined number of the divided images based on the situation information of the road surface reflected in the divided images included in the route.
• condition information of the road surface shown in the input image is calculated, and a route is evaluated according to the condition information. Therefore, a route can be evaluated taking into account conditions such as the undulations and friction of the road surface, and an appropriate route can be searched for.
• At least one of the functions of the calculation unit 121 and the evaluation unit 122 described above may be executed by an information processing device installed and connected anywhere on the network, that is, they may be executed by so-called cloud computing.
• Non-transitory computer readable medium includes various types of tangible storage medium.
• Examples of non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R/W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).
• the program may also be supplied to a computer by various types of transitory computer readable medium. Examples of transitory computer readable medium include electrical signals, optical signals, and electromagnetic waves.
• the temporary computer-readable medium can provide the program to the computer via a wired communication path, such as an electric wire or optical fiber, or via a wireless communication path.
• (Appendix 1) a calculation unit that calculates, for each divided image obtained by dividing an image into a plurality of regions, situation information that indicates a situation of a road surface reflected in the divided image based on the divided image; an evaluation unit that evaluates a route formed on the image by connecting a predetermined set of the divided images based on the situation information of a road surface reflected in the divided images included in the route;
• a route evaluation device comprising: (Appendix 2) 2.
• a path evaluation device calculates, based on the divided image, undulation condition information that represents an undulation condition of a road surface reflected in the divided image, as the undulation method; The evaluation unit evaluates the route based on the undulation information of a road surface shown in the divided images.
• Path evaluation device. (Appendix 3) 3.
• a path evaluation device according to claim 2, The calculation unit calculates an amount of undulation of a road surface reflected in the divided image as the undulation condition information.
• Path evaluation device. (Appendix 4) 3.
• the calculation unit calculates an undulating direction of a road surface reflected in the divided image as the undulating condition information.
• a path evaluation device evaluates the route including the divided image so that the evaluation is lower as the amount of undulation calculated for the divided image is larger.
• Path evaluation device. (Appendix 6) 3.
• the calculation unit calculates, based on the divided image, sliding property information that represents sliding property that represents how slippery a road surface is shown in the divided image, The evaluation unit evaluates the route based on the undulations and the sliding properties of a road surface shown in the divided images.
• a path evaluation device evaluates the route including the divided image so that the higher the value of the sliding property information calculated for the divided image is, the lower the evaluation of the route including the divided image is. Path evaluation device.
• a path evaluation device The calculation unit calculates, for each of the divided images, the situation information of a road surface reflected in the divided image based on information different from the image associated with a location corresponding to the divided image and the divided image. Path evaluation device. (Appendix 9) 2. A path evaluation device according to claim 1, the calculation unit calculates, for each of the new divided images in which the size of the area is changed, the situation information of a road surface reflected in the new divided image based on the new divided image; The evaluation unit evaluates the route formed on the image by connecting a predetermined one of the new divided images based on the situation information of a road surface reflected in the new divided image included in the route. Path evaluation device. (Appendix 10) 2.
• a path evaluation device evaluates the route based on moving object information indicating characteristics of a moving object moving along the route. Path evaluation device. (Appendix 11) 11. The path evaluation device according to claim 10, The evaluation unit evaluates the route based on the moving object information indicating a type of the moving object. Path evaluation device. (Appendix 12) 11. The path evaluation device according to claim 10, The evaluation unit evaluates the route based on the moving object information representing physical characteristics of the person who is the moving object. Path evaluation device. (Appendix 13) 2.
• a path evaluation device a display unit that displays a region image including a plurality of divided region images corresponding to the plurality of divided images, and displays the route on the region image in association with the divided region images corresponding to the divided images included in the route, Path evaluation device.
• the path evaluation device 14
• the display unit displays information based on the evaluation result of the route.
• Path evaluation device. Appendix 15
• the display unit displays, on the divided area image, information corresponding to the situation information calculated for the divided image corresponding to the divided area image. Path evaluation device. (Appendix 16) 2.
• a path evaluation device a control unit that controls a moving operation of the moving object so that the moving object moves along the route at a location corresponding to the image, Path evaluation device.
• An output unit that outputs the route optimized for the person based on person information that indicates characteristics related to the person's movement and load information that indicates a load applied when moving on the route based on an evaluation result of the route; Path evaluation device.
• the calculation unit inputs the segmented image into a model generated by machine learning learning data in which the segmented image and the situation information are associated with each other, the model being generated by the machine learning learning data, and acquires the situation information output from the model.
• Path evaluation device (Appendix 19) calculating situation information representing a situation of a road surface reflected in each divided image obtained by dividing the image into a plurality of regions based on the divided image; evaluating a route formed on the image by connecting a predetermined set of the divided images based on the situation information of a road surface reflected in the divided images included in the route; Path evaluation methods.
• (Appendix 20) calculating situation information representing a situation of a road surface reflected in each divided image obtained by dividing the image into a plurality of regions based on the divided image; evaluating a route formed on the image by connecting a predetermined set of the divided images based on the condition of a road surface shown in the divided images included in the route;
• a computer-readable storage medium that stores a program for causing a computer to execute a process.

Landscapes

• Engineering & Computer Science (AREA)
• Radar, Positioning & Navigation (AREA)
• Remote Sensing (AREA)
• Automation & Control Theory (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Traffic Control Systems (AREA)

Priority Applications (2)

Applications Claiming Priority (1)

Publications (1)

Family

ID=94533729

Family Applications (1)

Country Status (2)

Citations (4)

• 2023
  • 2023-08-04 JP JP2025538919A patent/JPWO2025032633A1/ja active Pending
  • 2023-08-04 WO PCT/JP2023/028556 patent/WO2025032633A1/ja active Pending

Patent Citations (4)

Also Published As

Similar Documents

Legal Events