JP7232613B2 - Track builder for car toy with camera - Google Patents

Description

TECHNICAL FIELD The present invention relates to a travel path construction tool suitable for a novel camera-equipped car toy that performs automatic driving and driving assistance based on images from an in-vehicle camera.

In recent years, in the field of actual automobiles, various technologies for performing speed control and steering control based on images captured by onboard cameras have been developed and put into practical use for automatic driving and driving assistance (for example, Patent Document 1). reference).

In the field of toy cars as well, the applicant company has been earnestly conducting research to provide car toys with cameras that are equipped with automatic driving and driving assist functions.

In this type of car toy, mainly due to the balance between product price and manufacturing cost, unlike a real car, a high-definition on-board camera, a processor (CPU, GPU) capable of high-speed and large-capacity processing, and However, it is difficult to adopt complex and advanced image recognition software.

For this reason, on the side of the road that is the subject of photography by the in-vehicle camera, the information necessary for automatic driving and driving assistance is captured by the in-vehicle camera by compensating for the inadequacies of the in-vehicle camera, processor, and image recognition software. Ingenuity is required to make it possible to obtain easily based on

Japanese Patent Application Laid-Open No. 2007-011432

The present invention has been made in view of the above-mentioned technical background, and its purpose is to enable easy acquisition of information necessary for automatic driving and driving assistance based on road surface images from an in-vehicle camera. To provide a travel path building tool for a car toy with a camera attached to a car.

Still other objects and effects of the present invention should be easily understood by those skilled in the art by referring to the following description of the specification.

The above technical problems are solved by a travel path building tool for a camera-equipped toy car, a method for constructing a travel path for a camera-equipped toy car, a camera-equipped toy car system, and a computer program having the following configurations. can be done.
That is, the traveling path building tool for a camera-equipped toy car according to the present invention includes:
A travel path building tool for a car toy with a camera that performs automatic driving and driving assistance based on the image of an in-vehicle camera,
a substrate that provides a running surface for the car toy;
a roadway drawn on the running surface of the base,
The roadway is partitioned into a roadway area and a surrounding area by a pair of left and right shoulder lines,
The roadway area is partitioned into two or more driving lanes by one or more lane boundary lines, and each of the pair of left and right shoulder lines is identifiable as to which side of the road shoulder line is on the left or right. It is a thing attached with a special marker.
According to such a configuration, each of the pair of left and right shoulder lines is provided with a marker capable of identifying which side of the road shoulder line is on the right or left side. By recognizing the presence of a road shoulder on either the left or right side in relation to the marker as long as at least one marker exists in the roadway, and calculating the roadway area based on the road shoulder thus recognized, It is possible to easily recognize the running state (running position, running direction, etc.) of the own vehicle on the running surface provided by the running path construction tool. Therefore, when driving in one of the lanes, due to approaching a curve, or due to being placed diagonally with respect to the direction of the lane at the start of driving, the road shoulder on either the left or right from the predetermined search area of the camera field of view. Even if the line disappears, as long as at least one road shoulder line remains within the predetermined search area, it is possible to recognize from the markers attached to it whether it is on the left or right side of the road shoulder line. Based on this, the relative relationship between the entire roadway and the own vehicle can be recognized, and automatic driving and driving assistance can be performed.

In a preferred embodiment, the markers may be linear markers extending on the opposite side of the roadway along each of the pair of left and right shoulder lines.
According to such a configuration, the markers that can identify the left or right side of the shoulder line are continuous along the roadway without interruption. You will not lose sight of the road shoulder line identification marker from the road. Moreover, since the linear marker extends on the side opposite to the roadway, it does not affect the roadway area.

In a preferred embodiment, the roadway area is colored in a first color, the lane boundary line and the pair of shoulder lines are colored in a second color, and the linear markers are colored in a third color. Further, a linear gap area may be provided between the linear marker and the shoulder line, and the linear gap area may be colored in the same first color as the roadway area. .
According to such a configuration, while scanning the captured image of the vehicle-mounted camera along the scanning line, the color arrangement pattern portion where the first color region exists on both sides of the second color region is detected. A pair of road shoulder line and lane boundary line is reliably recognized by a simple algorithm of searching and determining whether a third color area exists before or after such a color arrangement pattern point. becomes possible.

In a preferred embodiment, the roadway peripheral area may also be colored in the same first color as the roadway area.
According to such a configuration, even if the driver accidentally runs out of the roadway, both the second color area (road shoulder line and lane boundary line) and the third color area appear in the field of view of the vehicle-mounted camera. Since there is no (linear marker) either, it is possible to immediately recognize that the driver has deviated from the course.

In a preferred embodiment, among the first color, the second color, and the third color, the first color has the lowest lightness, and the second color may be the color with the highest lightness, and the third color may be a color having a lightness between the dark color and the light color.

According to such a configuration, by arranging the areas of the second color on both sides of the area of the first color, it is possible to make the road shoulder line and the lane boundary stand out on the image of the vehicle-mounted camera. can be done.

In a preferred embodiment, the first color may be black, the second color may be white, and the third color may be gray.
According to such a configuration, a color scheme similar to a real roadway is obtained, and even though it is a toy roadway, the sense of reality increases, and the presence of a gray area adjacent to a black area produces a so-called gradation effect. This also has the effect of making the marker lines inconspicuous visually.

In a preferred embodiment, the line width of the lane boundary line and the pair of road shoulder lines are both a narrow first line width, and the line width of the linear marker is The second line width may be sufficiently wider than the first line width.
According to such a configuration, when the linear marker is seen obliquely when approaching a curve while traveling along one of the lanes, the linear marker can be viewed even when the apparent line width becomes narrower. It becomes difficult to lose sight of it.

In a preferred embodiment, the roadway may be a circular roadway having a predetermined circular locus.
According to such a configuration, even if either the left or right shoulder line disappears from the predetermined search area in the field of view of the camera when approaching a curve, the car toy with the camera can continue to run in the determined lane. system can be provided.

In a preferred embodiment, the running surface may have a surface texture that suppresses reflection so as to avoid surrounding objects from being reflected in the camera.
According to such a configuration, it is possible to avoid the light from the lighting in the room being reflected on the road surface, reflected in the field of view of the vehicle-mounted camera, and affecting automatic driving and driving assistance as a disturbance.

In a preferred embodiment, the substrate may be a sheet-like or thin plate-like substrate.
According to such a configuration, for example, it can be conveniently carried by being folded or separated into small pieces.

Viewed from another aspect, the present invention can also be understood as a method of constructing a running path on the floor for a camera-equipped car toy that performs automatic driving or driving assistance based on images from an in-vehicle camera.
That is, in this construction method, a roadway area is defined by drawing a pair of left and right shoulder lines parallel to each other on the floor surface, and the roadway area is partitioned by one or more lane boundary lines. defines two or more driving lanes, and further, attaches a marker to each of the pair of left and right shoulder lines for identifying which side of the shoulder line the left and right shoulder lines belong to. be.
According to such a configuration, for example, by constructing a running path that exhibits the above-described effects on the floor surface of the room, an amusement place such as an indoor race circuit can be realized.

In a preferred embodiment, the markers may be linear markers extending on the opposite side of the roadway along each of the pair of left and right shoulder lines.
According to such a configuration, for example, in the above-described indoor race circuit, the marker that can identify the left or right side of the shoulder line is continuous along the roadway without interruption, so that the marker can be placed on either lane. While driving along the road, you will not lose sight of the marker for roadside line identification from the image of the vehicle-mounted camera. Moreover, since the linear marker extends on the side opposite to the roadway, it does not affect the roadway area.

Viewed from another aspect, the present invention can also be understood as a car toy system with a camera. That is, this camera-equipped car toy system
one of the track construction tools described above;
a camera-equipped car toy equipped with power for running and steering and an on-vehicle camera;
Based on the markers in the image of the vehicle-mounted camera, the running state of the own vehicle on the running surface provided by the running path building tool is recognized, and the running and steering markers are selected according to the desired running state. and a control unit that realizes automatic driving and driving assistance by controlling the power of the vehicle.

According to such a configuration, each of the pair of left and right shoulder lines is provided with a marker capable of identifying which side of the road shoulder line is on the right or left side. By recognizing the presence of a road shoulder on either the left or right side in relation to the marker as long as at least one marker exists in the roadway, and calculating the roadway area based on the road shoulder thus recognized, It is possible to recognize the running state (running position, running direction, etc.) of the own vehicle on the running surface provided by the running path construction tool. Therefore, when driving in one of the lanes, due to approaching a curve, or due to being placed diagonally with respect to the direction of the lane at the start of driving, the road shoulder on either the left or right from the predetermined search area of the camera field of view. Even if the line disappears, as long as at least one road shoulder line remains within the predetermined search area, it is possible to recognize from the markers attached to it whether it is on the left or right side of the road shoulder line. Based on this, the relative relationship between the entire roadway and the own vehicle can be recognized, and automatic driving and driving assistance can be performed.

According to a preferred embodiment, the control unit comprises an in-vehicle control device and a remote control device that are capable of wireless communication with each other,
The in-vehicle control device includes:
means for transferring an image acquired from the in-vehicle camera to the remote control device;
By executing the control command sent from the remote control device, the power for driving and steering is controlled to realize automatic driving and driving assistance.
The remote side control device includes:
By detecting both or one of the pair of left and right shoulder lines based on the markers in the image sent from the vehicle-mounted control device, and calculating the roadway area based on the thus detected road shoulder lines, means for recognizing the running state of the own vehicle on the running surface provided by the running path building tool;
means for generating a command indicating a correction amount for driving and steering necessary to bring the recognized driving state of the own vehicle to a desired driving state, and transmitting the command to the vehicle-mounted control device. It may be
According to such a configuration, by limiting the function to the image transfer function and the command execution function, the functions of the in-vehicle control device can be simplified. For advanced functions such as functions and command generation functions, by putting the burden on the remote control device, which can be realized by incorporating a dedicated application into a smartphone or tablet terminal, etc., the entire system can be used without significantly reducing the processing function. It is possible to reduce costs as

In a preferred embodiment, the road construction tool has a roadway area with a color scheme consisting of the first to third color areas, a lane boundary line, and left and right shoulder lines,
means for recognizing the running state of the own vehicle,
When detecting both or one of the pair of left and right shoulder lines, a predetermined search area within the field of view of the camera corresponding to a limited road surface area located at a predetermined distance in front of the vehicle is searched along scanning lines at predetermined intervals. , the presence of a third color region adjacent to either the left or the right side of the color array pattern in which the first color region exists on both sides of the second color region. A process for determining that it is a marker indicating a road shoulder line on the side may be executed.
According to such a configuration, while scanning the captured image of the vehicle-mounted camera along the scanning line, the color arrangement pattern portion where the second color region exists on both sides of the first color region is detected. A pair of road shoulder line and lane boundary line is reliably recognized by a simple algorithm of searching and determining whether a third color area exists before or after such a color arrangement pattern point. becomes possible.

Viewed from another aspect, the present invention is for causing a computer, which is an information processing device (for example, a smartphone, a tablet terminal, a notebook computer, a game machine, etc.) to function as the remote side control device in the car toy system with a camera described above. can also be grasped as a computer program of

According to the traveling road construction tool of the present invention, each of the pair of left and right shoulder lines is provided with a marker capable of identifying which side of the road shoulder line is on the left or right side. As long as at least one of the markers exists within the predetermined search area, the presence of the left or right shoulder line is recognized in relation to the marker, and the roadway area is calculated based on the recognized road shoulder line. Thus, it is possible to recognize the running state (running position, running direction, etc.) of the own vehicle on the running surface provided by the running path construction tool. Therefore, when driving in one of the lanes, due to approaching a curve, or due to being placed diagonally with respect to the direction of the lane at the start of driving, the road shoulder on either the left or right from the predetermined search area of the camera field of view. Even if the line disappears, as long as at least one road shoulder line remains within the predetermined search area, it is possible to recognize from the markers attached to it whether it is on the left or right side of the road shoulder line. Based on this, it is possible to recognize the relative relationship between the entire roadway and the own vehicle and execute automatic driving and driving assistance.

FIG. 1 is an external perspective view of a camera-equipped toy car circuit. FIG. 2 is an external perspective view of a camera-equipped toy car. FIG. 3 is a side view of the toy car with camera and camera field of view. FIG. 4 is a plan view showing the circuit running state of the camera-equipped toy car. FIG. 5 is an explanatory diagram of the camera field of view while traveling in the right lane. FIG. 6 is an explanatory diagram of the camera field of view when approaching a right curve. FIG. 7 is an explanatory diagram showing lane setting for two-lane one-way traffic. FIG. 8 is an explanatory diagram of a state in which the camera-equipped toy car is placed in a direction different from the lane. FIG. 9 is an explanatory diagram of a stand-type sign board. FIG. 10 is an explanatory diagram of the automatic overtaking mode. FIG. 11 is a configuration diagram of a camera-equipped car toy system. FIG. 12 is a configuration diagram of an in-vehicle control system. FIG. 13 is an explanatory diagram showing the relationship between scanning lines and road shoulders. FIG. 14 is a flow chart showing the operation of the in-vehicle control unit. FIG. 15 is a flowchart showing the operation of the smartphone-side application. FIG. 16 is a detailed flowchart of the running state recognition process.

Appended below is a preferred embodiment of a travel path construction tool for a camera-equipped toy car, a method for constructing a travel path for a camera-equipped toy car, a camera-equipped toy car system, and a computer program according to the present invention. A detailed description will be given with reference to the drawings.

[About the track construction tool]
FIG. 1 shows an external perspective view of a camera-equipped toy car circuit 1, which is an embodiment of a running path building tool for a camera-equipped toy car according to the present invention.

As shown in the figure, this camera-equipped toy car circuit 1 has a substrate (substrate) 11 of rectangular outline. The substrate 11 provides a running surface for the camera-equipped car toy 2, and can be made of cardboard, thin resin plate or sheet, or the like. The substrate is not limited to a plate-like body, but may be a table-like or box-like one whose upper surface is provided as a running surface.

The substrate 11 can be separated into a plurality of small pieces or folded into a small size by appropriately providing a mountain fold line or a valley fold line so as to be convenient to carry when selling at a toy store or the like or by mail order. It may be possible. The substrate 11 shown in the figure can be separated into a plurality of rectangular pieces, and can be separated from each other in the manner of a jigsaw puzzle when the pieces are laid out.

In the illustrated example, the substrate 11 is surrounded on its four sides by four surrounding wall plates 12a, 12b, 12c and 12d. The inner surfaces of the surrounding walls 12a, 12b, 12c, and 12d are configured to have a uniform color (for example, white) to prevent objects unnecessary for travel control from appearing in the field of view of the vehicle-mounted camera. In order to avoid unnecessary reflection of objects, the surrounding walls 12a, 12b, 12c, and 12d can be removed by taking measures such as providing a dead area on the light receiving surface of the image sensor on the image processing side. can also

The surface of the substrate 11 is drawn with a roadway having an elliptical or oval circular locus. The illustrated roadway is partitioned into a roadway area and a peripheral area by a pair of left and right shoulder lines (an inner shoulder line 16 and an outer shoulder line 17).

The roadway area is defined by one or more lane boundary lines (one lane boundary line 15 in this example) and two or more driving lanes (in this example, two lanes consisting of an inner lane 13 and an outer lane 14). Further, each of the pair of left and right shoulder lines (the inner shoulder line 16 and the outer shoulder line 17) is provided with a marker (inner marker line 18) that can identify which side of the road shoulder line is on the left or right side. and outer marker lines 19).

According to this configuration, each of the pair of left and right shoulder lines is provided with a marker (the inner marker line 18 and the outer marker line 19) that can identify which side of the road shoulder line 16 or 17 is on the left or right side. Therefore, as long as at least one of the markers exists within the predetermined search area 42 within the field of view of the vehicle-mounted camera, the presence of the shoulder line on either the left or right side is recognized in relation to that marker, and thus recognized. By calculating the roadway area with reference to the road shoulder, it is possible to recognize the running state (running position, running direction, etc.) of the own vehicle on the running surface provided by the running road construction tool. Therefore, when the vehicle is approaching a curve while driving in either lane 13 or 14 (see FIG. 6), or when it is placed diagonally with respect to the direction of the lane at the start of driving (see FIG. 8). Even if either the left or right shoulder line disappears from the predetermined search area in the field of view of the camera, as long as at least one shoulder line 16 or 17 remains within the predetermined search area 42, the marker 18 attached to it or From 19, it is possible to recognize whether it is the left or right side of the shoulder line, so based on this, the relative relationship between the entire roadway and the own vehicle is recognized, and automatic driving and driving assistance are performed. can be executed.

Various modes can be adopted as markers to be attached to the inner shoulder line 16 and the outer shoulder line 17 . For example, the inner shoulder line 16 and the outer shoulder line 17 may have different colors or densities, different line widths, or a specially designed identification mark attached on the shoulder line. can take any form.

In particular, in the illustrated example, the marker lines 18 and 19 are linear markers extending along the pair of road shoulder lines 16 and 17 on the side opposite to the roadway. According to such a linear marker, since the marker that can identify which side of the shoulder line 16, 17 is on the left or right is continuous along the roadway, the vehicle is traveling along either lane. , the marker for roadside line identification is not lost from the image of the vehicle-mounted camera. Moreover, since the linear marker extends on the side opposite to the roadway, it does not affect the roadway area.

The above-mentioned roadway area is colored in a first color, the lane boundary line 15 and the pair of shoulder lines 16, 17 are colored in a second color, and the inner marker line 16 and outer marker line 17 are colored in a third color. Further, linear gap regions are provided between the inner marker line 18 and the inner shoulder line 16 and between the outer marker line 19 and the outer shoulder line 17, respectively. The area is colored in the same first color as the roadway area.

According to such a configuration, while scanning the captured image of the vehicle-mounted camera along the scanning line, the color arrangement pattern portion where the first color region exists on both sides of the second color region is detected. A pair of road shoulder line and lane boundary line is reliably recognized by a simple algorithm of searching and determining whether a third color area exists before or after such a color arrangement pattern point. becomes possible.

In the illustrated example, other areas (ground color area 11a) including the area around the roadway are also colored in the same first color as the roadway area. In this case, neither the second color area (the road shoulder lines 16, 17 and the lane boundary line 15) nor the third color area (the inner marker line 18 and the outer marker line 19) exist in the field of view of the vehicle-mounted camera. , can be made to immediately recognize that it has deviated from the course.

The configuration of the roadway area described above has been devised in various ways also from the viewpoint of color scheme. That is, among the first color, the second color, and the third color, the first color is the color with the lowest lightness (dark color), and the second color is the color with the highest lightness ( light color), and the third color is colored to be a color having a lightness between the dark color and the light color.

According to such a configuration, by arranging the areas of the second color on both sides of the area of the first color, road shoulders and lane boundaries can be made more conspicuous on the image of the vehicle-mounted camera. , can improve their recognition accuracy.

In particular, in the illustrated example, the first color is black, the second color is white, and the third color is gray. Therefore, even though it is a toy road, it is more realistic, and the presence of the gray area adjacent to the black area makes it easy for the machine eye to recognize, but the human eye can easily recognize it due to the so-called gradation effect. has the effect of making the marker lines inconspicuous .

The configuration of the roadway area described above is also devised from the viewpoint of the line width. That is, the line width of the lane boundary line 15 and the pair of road shoulder lines 16 and 17 are both set to a narrow first line width, and the line widths of the inner and outer marker lines 18 and 19 are The second line width is sufficiently wider than the first line width.

According to such a configuration, while traveling along the inner lane 13 or the outer lane 14, when approaching a curve and obliquely looking at the inner marker line 18 or the outer marker line 19, the apparent line width becomes Since the marker lines 18 and 19 have a sufficient line width even when the line width is narrowed, it is difficult to overlook even if the line width becomes thin.

In the illustrated example, the roadway is a circular roadway having a predetermined circular locus, but the application of the present invention is not limited to such an elliptical or oval circular roadway. Of course, it may be an upper roadway or an S-shaped roadway. However, in such a circular road, even if either the left or right shoulder line disappears from the predetermined search area of the camera's field of view due to approaching a curve, the inner marker line 18 or the outer marker line 19 can be recognized. As long as there is, it is possible to continue traveling in the determined lane.

In addition, as described above, the traveling surface, which is the surface of the substrate 11, has a surface property (for example, a fine rough surface) that suppresses reflection to avoid surrounding objects from being reflected in the camera. It is possible to prevent the light from the lighting from reflecting on the road surface and being reflected in the field of view of the in-vehicle camera and affecting automatic driving and driving assistance as a disturbance.

[About car toys with camera]
Next, the appearance, electrical hardware configuration, and software configuration of the camera-equipped car toy will be described in detail.

FIG. 2 is a perspective view showing an example of the appearance of a camera-equipped car toy, FIG. 3 is a side view of the car toy and the field of view of the camera, and FIG. It is

As shown in FIG. 2, the camera-equipped toy car 2 includes a steering actuator 25d (see FIG. 12) for driving the steering mechanism of the front wheels 2c and a steering actuator 25d (see FIG. 12) for driving the steering mechanism of the front wheels 2c. A running motor 25c (see FIG. 12) for operating the rotation drive mechanism of the rear wheel 2d is accommodated. Alternatively, two motors that can be driven independently may be provided as the running motors, and a difference in rotational speed may be provided between the motors so that the rotational movement is realized by utilizing the reaction from the floor surface.

As shown in FIG. 3, a camera 2e is attached to the front of the vehicle body 2a with its field of view directed forward. As is well known to those skilled in the art, the camera 2e includes an optical system for collecting and capturing light from the front and an optical system for coupling the light captured through the optical system to its imaging surface. an image sensor (eg, a CMOS image sensor, a CCD image sensor, etc.) located in the . It should be noted that the appearance of the illustrated camera is merely a conceptual diagram, and is not an exact imitation of the actual camera.

Although not shown, in the above-mentioned vehicle interior space, in addition to LEDs that make up the headlights and speakers for generating various sounds (warning sounds, chattering sounds, etc.), various devices that make up the on-vehicle control system A circuit board on which circuit parts are mounted, a rechargeable/non-rechargeable battery that constitutes a power source, a supercapacitor that constitutes a power source, and the like are accommodated.

The electrical hardware configuration of the vehicle-mounted control system will be described in detail with reference to FIG. As shown in the figure, the in-vehicle control system is mainly composed of a microcomputer having a microprocessor (MPU) 21, a ROM 22 and a RAM 23 as constituent elements. In particular, when high-speed image processing is required, the microcomputer described above may be provided with a GPU dedicated to image processing. As is well known to those skilled in the art, a microprocessor (MPU) 21 implements various functions required as an in-vehicle control device by executing a system program stored in a ROM 22 using a RAM 23 as a work area. It is something to do. Furthermore, when setting the machine number, a micro DIP switch for setting on the vehicle-mounted control device or a flash ROM (FROM) for setting from the remote control device may be provided.

An input I/F section 24, an output I/F section, and a wireless communication section 26 are connected to the system bus of the microcomputer, as shown. The input I/F unit 24 is used for taking in captured images from the vehicle-mounted camera 24a (2e). The output I/F unit 25 drives the LEDs that make up the headlights 25a, drives the speaker 25b for sound output, drives the driving motor 25c that is used for forward/backward movement and speed control, and furthermore, drives left and right turns. It is used when driving the steering actuator 25d. The wireless communication unit 26 is used for wireless communication between the in-vehicle control device and the remote control device (details will be described later).

FIG. 14 shows a flow chart showing the operation of the vehicle-mounted controller realized by executing the system program stored in the ROM 22. As shown in FIG. As shown in the figure, after power-on, the in-vehicle control device initializes various necessary flags and registers by initialization processing (step 101), and then performs photographing processing (image acquisition) ( Step 101) and a process of transmitting the acquired image to the smart phone constituting the remote side control device (step 102) are executed, and after that, waiting for arrival of any command from the remote side control device (step 104). , 105 YES), and by cyclically executing the process (step 106) for performing the process corresponding to the received command, various functions required for the vehicle-mounted control device are realized.

These functions include, for example, 1) a function of operating the steering actuator 25d so that steering is performed by a steering amount specified by a command, 3) a function of operating the LEDs constituting the headlights so as to turn on/off the lights specified by commands; 4) a function of generating warning sounds and chattering sounds specified by commands; 5) A function to execute an overtaking sequence specified by a command to avoid or overtake preceding vehicles and obstacles; 6) A function to execute a shoulder line search sequence specified by a command. , a function of searching for road shoulders, and the like.

[Regarding the remote controller]
Next, the remote control device will be described in detail. The remote-side control device is mainly responsible for image recognition processing and operation amount calculation processing for error correction. It can be configured by installing an application program (app) dedicated to the toy system. Examples of the stored program type information processing apparatus having a communication function include smart phones, tablet personal computers, notebook personal computers, various game machines, and the like.

FIG. 11 shows the front appearance of a smartphone 3 configured to function as a remote side control device by installing a dedicated application. The smart phone 3 is capable of wireless communication with the in-vehicle control device (see FIG. 12) of the car toy 2 with camera.

In the figure, 31 is a push button on the main body side, 32 is a display screen using liquid crystal or organic EL, for example, 33 is a microphone on the main body side, and 34 is a camera on the main body side. Also, 35 is a slide operator display for steering operation, 36 is a slide operator display for switching between forward and backward movement and speed adjustment, 37 is an illuminated automatic driving button display for switching to automatic operation mode (AT), 38 is an illuminated assist driving button display for switching to the assisted driving mode (AST), and 39 is an illuminated manual driving button display for switching to the manual driving mode (ML). The operator displays 35, 36 and button displays 37 to 39 are all displayed on the display screen 32 by software, and are operated via a touch panel attached to the surface of the display screen 32. is configured to Note that the basic hardware configuration of this type of smartphone is well known to those skilled in the art, and therefore descriptions using hardware block diagrams and the like will be omitted.

Next, the configuration of the application dedicated to the car toy system with camera will be described in detail with reference to FIGS. 15 and 16. FIG. A general flowchart showing the operation of the smartphone side application is shown in FIG. 15, and a detailed flowchart of the running state recognition processing in the same flowchart is shown in FIG.

In this camera-equipped car toy system, prior to the start of play, several items such as "lane setting", "running number setting", "obstacle response setting" and "driving mode setting" are performed. , the selection setting operation is performed for the setting items.

Here, "lane setting" refers to a circular road having two parallel lanes (the inner lane 13 and the outer lane 14) shown in FIG. It sets how it is used. At this time, as options for setting, for example, [1] use a two-way road with one lane on one side for left-hand traffic, [2] use a two-way road with one lane on one side for right-hand traffic (see Fig. 4). ), [3] use as a two-lane one-way road as a clockwise loop road, and [4] use as a two-lane one-way road as a counterclockwise loop road (see Fig. 7). be done.

"Setting the number of cars running" is for setting the number of car toy cars with cameras to run on the premise of the lane setting described above. At this time, setting options include, for example, [1] 1-car travel, [2] 2-car travel, [3] 3-car travel, [4] 4-car travel, and so on. In this example, one lane is basically set to allow a plurality of camera-equipped toy cars to run.

The "setting for response when an obstacle exists" is for setting how to respond when an obstacle such as a preceding vehicle, a stopped vehicle, or a passerby exists in front of the same lane. At this time, setting options include [1] stop immediately before, [2] take a detour or overtake in a prescribed steering sequence, [3] run while following.

"Operating mode setting", as the name suggests, sets the operating mode. At this time, setting options include [1] automatic operation mode, [2] assist operation mode, [3] manual operation mode, and the like.

In addition, a setting mode is separately provided on the dedicated application side for setting operations related to each setting item described above, and in this setting mode, various settings displayed on the display screen 32 of the smartphone 3 By following the guide, setting operations can be easily performed for each item.

In addition, when setting the number of running cars, either a unique machine number is set in advance by a micro DIP switch, for example, on each car toy side, or communication is performed by a polling selection method between a smartphone and each car toy. Via, a process such as allocating the machine number and storing it in the flash ROM of the machine is performed.

Suppose now that "lane setting" is [2] (two-way road with one lane on one side is used for right-hand traffic (see Fig. 4)), "number of vehicles setting" is [3] (three vehicles running), "Response setting when an object exists" is set to [2] (detour or overtake in a prescribed steering sequence (see FIG. 10)), and "driving mode setting" is set to [1] (automatic driving mode). .

In this state, when the process shown in FIG. 15 is started, in the initialization process (step 201), various flags and registers required for calculation are initialized, and then the machine number [n] is set to the initial value. (step 202), and image reception processing (step 203) from the car toy 2 corresponding to the n-th machine is executed.

Subsequently, a driving state recognition process (step 204) is executed based on the image thus received. This running state recognition processing (step 204) is to determine where and how the toy car 2 with the car number [n] is running on the camera-equipped toy car circuit 1 shown in FIG. The driving state recognized in this way is used for correction processing of driving and steering for automatic driving and assisted driving. Then, in this running state recognition process (step 204), the configuration (the inner marker line 18 and the outer marker line 19), which is the main part of the present invention, functions effectively.

An example of the driving state recognition process (step 204) is shown in detail in FIG. In the figure, when the process is started, a lane search area setting process (step 301) is executed. This lane search area is a horizontally long rectangular area indicated by reference numeral 42 in FIGS. It corresponds to the area. This lane search area 42 includes both the inner marker line 18 and the outer marker line 19 during normal lane travel, and even if the direction of the lane and the direction of the vehicle are slightly different, , at least one of those marker lines 18, 19 is arranged to be included (see FIG. 5).

Next, within the lane search area 42, along scanning lines (Lk, Lk+1, Lk+2, Lk+3 . ("black"→"white"→"black") is executed (see FIG. 13).

At this time, since the marker lines 18 and 19 are linear bodies, there is no possibility that the marker lines 18 and 19 will disappear in any of the image frames (for example, 30 frames/second). For the same reason, even if the interval between the scanning line Lk and the scanning line Lk+1 is made sufficiently wider than the pixel size in order to increase the search processing speed, some of the pixels forming the marker lines 18 and 19 There is no danger of losing sight of the (see FIG. 13).

Here, the pixel value array pattern (“black”→“white”→“black”) is a pixel value array pattern that changes from a series of black pixel states to a series of white pixel states and again to a series of black pixel states. Assuming the toy car circuit 1 shown in FIG. There can be no other than when crossing the lane boundary line 15 or when crossing the inner shoulder line 16 (see FIG. 13(b)).

After checking the pixel value array pattern along the k-th scanning line Lk (step 303), if it is determined that there is an array pattern (“black”→“white”→“black”) (step 304 YES), Next, whether there is a pixel value "gray" immediately before and adjacent to such a pixel value array pattern, whether there is a pixel value "gray" adjacent immediately after, and whether there is a pixel value "gray" immediately before or immediately after such a pixel value array pattern A determination is made whether "ash" is not present (step 305).

Here, when it is determined that the pixel value "gray" exists immediately before and adjacent to the pixel value array pattern ("black"→"white"→"black") (step 305 "immediately adjacent"), The horizontal coordinate value of the white pixel positioned at the center of the series of "white" pixel rows forming the pixel value array pattern is temporarily stored as a candidate for the inner shoulder line 16 (step 306).

If it is determined that the pixel value "gray" is immediately adjacent to the pixel value array pattern ("black"→"white"→"black") (step 305 "immediately adjacent"), the pixel value array The horizontal coordinate value of the white pixel located at the center of the series of "white" pixel columns forming the pattern is temporarily stored as a candidate for the outer shoulder line 17 (step 308).

If it is determined that the pixel value "gray" does not exist immediately before or after the pixel value array pattern ("black"→"white"→"black") (step 305 "no adjacent"), the pixel The horizontal coordinate value of the white pixel located at the center of the series of "white" pixel columns forming the value array pattern is temporarily stored as a lane boundary line 15 candidate (step 307).

Even if the pixel value array pattern is checked (step 303), if it is determined that the pixel value array pattern (“black”→“white”→“black”) does not exist (step 304 NO), scanning line number k is updated by +1 (step 311), the above series of processes (steps 303 to 308) are repeated, and after the scanning line number k reaches a predetermined maximum value, the validity determination process (step 310) is performed. and migrate.

In this validity determination process (step 310), for each of the inner shoulder line 16, the outer shoulder line 17, and the lane boundary line 15, the degree of continuity in the vertical direction is considered for a series of temporarily stored candidates. , it is determined that the search for each of the inner shoulder line 16, the outer shoulder line 17, and the lane boundary line 15 is successful. Regarding the continuity in the vertical direction, for example, the deviations between the tentatively determined candidate coordinates are sequentially obtained for two adjacent scanning lines (Lk, Lk+1), and it is determined that the deviations are substantially constant. Based on this, it can be determined that those candidate coordinates are arranged in order on a certain line.

Subsequently, if the continuity of a series of candidates for any of the inner shoulder line 16, the outer shoulder line 17, and the lane boundary line 15 is recognized and it is determined that the search is successful (step 312 YES), the inner shoulder line 16 , the outer road shoulder 17 and the lane boundary 15, which are determined to be effective (search success), and the vehicle's reference position 44 (see FIGS. 5 and 6) to recognize the running state of the vehicle. The process (step 313) is executed, and upon completion of the execution, recognition success is stored (step 314), and the process ends. On the other hand, if it is determined that the search fails because the continuity of a series of candidates is not recognized for any of the inner shoulder line 16, the outer shoulder line 17, and the lane boundary line 15, recognition failure is stored. (Step 315), the process ends.

According to the above processing (steps 304 to 308), by providing the inner marker line 18 and the outer marker line 19, white lines corresponding to the inner shoulder line 16, the lane boundary line 15 and the outer shoulder line 17 are drawn. Pixels can be directly recognized as "inner shoulder", "lane boundary" and "outer shoulder", rather than just "lines". Therefore, if any one of them is recognized, the entire road surface situation can be calculated from known information (for example, the positional relationship between these "lines", the lane width, the mounting posture of the in-vehicle camera, etc.). It can be easily guessed, and based thereon, it is possible to accurately recognize the driving situation of the own vehicle (for example, where in the circuit 1 and in which direction the vehicle is running).

Returning to the flow chart of FIG. 15, when it is determined that the running state of the own vehicle has been successfully recognized (step 205), the process of generating and transmitting an error correction command for running in the target lane (step 206) is executed. In this process (step 206), the driving state of the vehicle recognized in the previous process (step 204) and the target driving lane (for example, inner lane 13) determined by the "lane setting" are set. Based on this, the amount of correction of running/steering required to guide the own vehicle to the center line 43 (see FIG. 5) of the target lane is calculated, and this is sent to the n-th car toy 2 as a running control command for error correction. wirelessly transmitted as

Subsequently, the process shifts to generation/transmission processing of other running control commands (step 208). In this process (step 208), a traveling control command necessary for overtaking control, traffic sign control, etc. is transmitted.

That is, in this embodiment, as shown in FIG. 9, several types of stand type signboards 51 are prepared. Each of the sign plates 51 has an L-shaped configuration in a side view including an upright portion 51a and a seat portion 51b, and can be arranged on the circuit 1 in a self-supporting state. A square marker 51c composed of a thick square is drawn on the upper portion of the upright plate portion 51, and a sign figure 51d corresponding to a traffic sign is drawn in the inner area surrounded by the marker 51c. .

When such a stand-type sign plate 51 is placed on the running surface of the circuit 1, a sign plate image 45 appears within the field of view 41 of the vehicle-mounted camera, as shown in FIG. A square image corresponding to the square marker 51c of the sign plate image 45 can be easily searched by pattern recognition processing using the corresponding square mask. Moreover, once the square marker 51c is found, the marker figure 51d drawn inside it can also be easily determined by pattern recognition processing.

Therefore, the stop operation is performed for the marker graphic 51d of FIG. 9(a), the entry prohibition operation is performed for the marker graphic 51c of FIG. 9(b), and the speed is 60 per hour for the marker graphic 51c of FIG. As in the case of a kilometer-equivalent travel, the sign plate 51 and the corresponding driving action are associated in advance in the table, so that just by arranging some sign plate 51 on the traveling surface, the camera can pass there. Appropriate driving instructions can be given to the toy car 2 .

Furthermore, in this example, the contents of the driving instruction are valid only when the sign plate 51 is placed outside the roadway area (the area sandwiched between the inner shoulder line 16 and the outer shoulder line 17). Like this, it is organized on the application side.

Therefore, as shown in FIG. 10, when the sign board 51 is placed beside the roadway, in the command generation/transmission process (step 208), the vehicle travels according to the meaning of the sign board 51 (at a speed of 60 km/h). While the travel control command is generated and transmitted so as to travel equivalent to kilometers), if it is located inside the roadway area (the area sandwiched between the inner shoulder line 16 and the outer shoulder line 17), It is simply recognized as an obstacle, and a travel control command corresponding to a predetermined obstacle handling process is generated and transmitted.

Furthermore, in this embodiment, in the command generation/transmission process (step 208), as shown in FIG. , it is also possible to generate and transmit a running control command for instructing an operation to overtake the preceding vehicle by a predetermined steering sequence according to the overtaking trajectory line 47 .

The running and control corresponding to the above signs and the running and control for automatic overtaking are both performed based on the running state of the own vehicle recognized in the running state recognition processing (step 204). In the present invention, the road shoulder line and lane boundary line recognition processing (steps 303 to 308), which forms the basis of the driving state recognition processing (step 204), can be performed quickly and reliably based on the existence of the marker lines 18 and 19. Therefore, it is possible to ensure the certainty of the operation corresponding to the sign and the automatic overtaking operation.

Next, when it is determined that the recognition has failed in the determination process (step 205) (step 205 NO), for example, as shown in FIG. First, in a state in which the speed of the own vehicle is set to a predetermined low speed, the direction of the own vehicle is steered by a predetermined amount, thereby turning the own vehicle in a minute radius while turning the surroundings. to search for either of the marker lines 18, 19, the appropriate drive/steer command is sent to the on-vehicle device (step 207). Thus, if either one of the marker lines 18, 19 is found during the marker line searching operation, the above-described series of processing (steps 206, 208) is returned to.

Note that the running and control for the marker search operation described above are all performed based on the running state of the own vehicle recognized in the running state recognition process (step 204). , the road shoulder line and lane boundary line recognition processing (steps 303 to 308), which forms the basis of the driving state recognition processing (step 204), can be performed quickly and reliably based on the presence of the marker lines 18 and 19. It is possible to secure the certainty of the operation operation.

In addition to the automatic driving mode, the application of this embodiment also has an assist driving mode and a manual driving mode, as described above. Briefly describing these, these modes can be switched by operating three push button displays 37 , 38 , and 39 displayed on the upper right of the display screen 32 of the smartphone 3 .

That is, in order to switch from the automatic operation mode to the manual operation mode, the manual travel button (ML) 39 is pressed. Then, the slide operator display 35 for the steering operation and the slide operator display 36 for the forward/backward operation drawn on the display screen 32 are activated and can be used.

In this state, while viewing the vehicle-mounted camera image drawn on the display screen 32, the slide operator 35 is operated left and right with the right hand and the slide operator 35 is operated with the left hand in consideration of the relationship with the vehicle position display 44. By operating 36 back and forth, it is possible to freely run on the circuit 1 at any speed.

Also, in order to switch from the automatic driving mode to the assist driving mode, the support driving button (SAST) is pressed. Then, as in the case of manual operation, the slide operator display 35 for steering operation and the slide operator display 36 for forward/backward operation drawn on the display screen 32 are activated and can be used.

In that state, while manually driving along the desired lane while visually recognizing the image of the in-vehicle camera drawn on the display screen 32, if a sign that it is about to leave the lane is detected, the same as in the automatic driving mode. Then, an error correction command for running in the target lane is generated and transmitted, and as a result, the vehicle runs without deviating from the target lane.

In addition, when a preceding vehicle or an obstacle appears in front of the target lane during manual driving, driving corresponding to the steering sequence for automatic overtaking is performed in the same manner as in the automatic driving mode. A control command generation/transmission process is executed, and as a result, the preceding vehicle or obstacle is automatically avoided at a desired speed.

Note that the driving and control for automatic lane following and the driving and control for automatic overtaking described above are both based on the driving state of the own vehicle recognized in the driving state recognition processing (step 204). In the present invention, the road shoulder line and lane boundary line recognition processing (steps 303 to 308), which forms the basis of the driving state recognition processing (step 204), is performed at high speed based on the presence of the marker lines 18 and 19. Moreover, since it can be performed reliably, it is possible to ensure the certainty of the lane follow-up operation and the automatic overtaking operation.

[Other embodiments]
Viewed from another aspect, the present invention can also be understood as a method of constructing a running path on the floor for a camera-equipped car toy that performs automatic driving or driving assistance based on images from an in-vehicle camera.

That is, in this construction method, a roadway area is defined by drawing a pair of left and right shoulder lines parallel to each other on the floor surface, and the roadway area is partitioned by one or more lane boundary lines. defines two or more driving lanes, and further, attaches a marker to each of the pair of left and right shoulder lines for identifying which side of the shoulder line the left and right shoulder lines belong to. be.

According to such a configuration, for example, by constructing a running path that exhibits the above-described effects on the floor surface of the room, an amusement place such as an indoor race circuit can be realized.

In a preferred embodiment, the markers may be linear markers extending on the opposite side of the roadway along each of the pair of left and right shoulder lines.

According to such a configuration, for example, in the above-described indoor race circuit, the marker that can identify the left or right side of the shoulder line is continuous along the roadway without interruption, so that the marker can be placed on either lane. While driving along the road, you will not lose sight of the marker for roadside line identification from the image of the vehicle-mounted camera. Moreover, since the linear marker extends on the side opposite to the roadway, it does not affect the roadway area.

In the above embodiment, the configuration of the control unit is composed of an in-vehicle control device (mainly specialized for image transfer and command execution) and a remote control device (mainly specialized for image recognition and correction value calculation). However, if the performance of the device is improved or the price is reduced in the future, it is also possible to integrate the above two control devices into a car toy with a camera.

INDUSTRIAL APPLICABILITY The present invention can be used in the toy manufacturing industry to provide a camera-equipped car toy system equipped with automatic driving and driving assist functions.

1 toy car circuit with camera 2 toy car with camera 2a car body 2b chassis 2c front wheel 2d rear wheel 2e camera 3 smart phone 11 substrate 11a background area 12a, 12b, 12c, 12d enclosure plate 13 inner lane 14 outer lane 15 lane boundary Line 16 Inner shoulder line 17 Outer shoulder line 18 Inner marker line 19 Outer marker line 21 MPU
22 ROMs
23 RAM
24 Input section 24a Camera 25 Output section 25a Headlight 25b Speaker 25c Running motor 25d Steering actuator 31 Main body side push button 32 Display screen 33 Main body side microphone 34 Main body side front camera 35 Slide operator display for steering 36 For forward and backward movement 37 Automatic run button display 38 Assist run button display 39 Manual run button display 41 Camera field of view 42 Search area 43 Inside lane center line 44 Vehicle running position display 45 Signboard image 46 Leading vehicle image 51 Stand type sign Plate 51a Standing plate portion 51b Seat portion 51c Square marker 51d Marking figure Lk, Lk+1, Lk+2, Lk+3 Scanning line

Claims (16)

A camera-equipped car toy equipped with power for running and steering and an on-board camera, and recognition of the running state of the own car on the running surface provided by the runway building tool based on the markers in the image of the on-board camera. and a control unit that realizes automatic driving and driving assistance by controlling the power for driving and steering according to a desired driving state . is a tool,
a substrate that provides a running surface for the car toy;
a roadway drawn on the running surface of the base,
The roadway is partitioned into a roadway area and a surrounding area by a pair of left and right shoulder lines, the roadway area is partitioned into two or more driving lanes by one or two or more lane boundary lines, and Each of the pair of left and right shoulder lines is provided with the marker capable of identifying which side of the shoulder line is on the left or right, respectively. 2. The travel path building tool for a camera-equipped toy car system according to claim 1, wherein said markers are linear markers extending along each of said pair of left and right shoulder lines on the side opposite to the roadway. The roadway area is colored in a first color, the lane boundary line and the pair of road shoulder lines are colored in a second color, and the linear markers are colored in a third color, and the linear markers are colored in a third color. 3. The camera-equipped automobile according to claim 2, wherein a linear gap area is provided between the road and the shoulder line, and the linear gap area is colored in the same first color as the road area. Track builder for toy systems . 4. The track builder for a camera-equipped toy car system according to claim 3, wherein said roadway peripheral area is also colored in the same first color as said roadway area. Among the first color, the second color, and the third color, the first color is the color with the lowest lightness, and the second color is the color with the highest lightness. and wherein said third color is a color having a brightness between said dark color and said light color. 6. The track builder for a camera toy car system of claim 5, wherein said first color is black, said second color is white, and said third color is gray. . Both the lane boundary line and the pair of road shoulder lines have a narrow first line width, and the line width of the linear marker is sufficiently wider than the first line width. 3. The track builder for a camera toy car system of claim 2, wherein the second line width is wider than the line width. 2. The track builder for a camera toy car system according to claim 1, wherein said roadway is a circuit roadway having a predetermined circuitous trajectory. 2. A traveling path construction tool for a camera-equipped toy car system according to claim 1, wherein said traveling surface has a surface texture that suppresses reflection so as to avoid surrounding objects from being reflected in the camera. 2. A traveling path construction tool for a camera-equipped car toy system according to claim 1, wherein said substrate comprises a sheet-like or thin plate-like substrate. A camera-equipped car toy equipped with power for driving and steering and an on-board camera, and based on the markers in the image of the on-board camera, recognize the driving state of the own vehicle on the road, and respond to the desired driving state. , a control unit that realizes automatic driving and driving assistance by controlling the power for driving and steering, and a method for constructing the driving path on the floor for a car toy system with a camera. hand,
A roadway area is defined by drawing a pair of left and right shoulder lines parallel to each other on the floor surface, and two or more driving lanes are created by partitioning the roadway area with one or more lane boundary lines. and further, each of the pair of left and right shoulder lines is provided with a marker for identifying which side of the shoulder line the left and right shoulder lines are. How roads are built. 12. Construction of a running path for a camera-equipped toy car system according to claim 11, wherein said markers are linear markers extending along each of said pair of left and right shoulder lines on the side opposite to said roadway. Method. A traveling path building tool according to any one of claims 1 to 10;
a camera-equipped car toy equipped with power for running and steering and an on-vehicle camera;
Based on the markers in the image of the vehicle-mounted camera, the running state of the own vehicle on the running surface provided by the running path building tool is recognized, and the running and steering markers are selected according to the desired running state. A car toy system with a camera, comprising a control unit that realizes automatic driving and driving assistance by controlling the power of the car. The control unit comprises an in-vehicle control device and a remote control device that are capable of wireless communication with each other,
The in-vehicle control device includes:
means for transferring an image acquired from the in-vehicle camera to the remote control device;
By executing the control command sent from the remote control device, the power for driving and steering is controlled to realize automatic driving and driving assistance.
The remote side control device includes:
By detecting both or one of the pair of left and right shoulder lines based on the markers in the image sent from the vehicle-mounted control device, and calculating the roadway area based on the thus detected road shoulder lines, means for recognizing the running state of the own vehicle on the running surface provided by the running path building tool;
means for generating a command indicating a correction amount for driving and steering necessary to bring the recognized driving state of the own vehicle to a desired driving state, and transmitting the command to the vehicle-mounted control device. 14. The camera-equipped toy car system of claim 13, wherein: The travel path building tool is according to any one of claims 3 to 6,
means for recognizing the running state of the own vehicle,
When detecting both or one of the pair of left and right shoulder lines, a predetermined search area within the field of view of the camera corresponding to a limited road surface area located at a predetermined distance in front of the vehicle is searched along scanning lines at predetermined intervals. , the presence of a third color region adjacent to either the left or the right side of the color array pattern in which the first color region exists on both sides of the second color region. 15. The camera-equipped toy car system according to claim 14, which performs a process of determining that the marker is a marker indicating a side road shoulder line. wherein the computer constitutes the remote controller in the camera-equipped toy car system of claim 14 ,
By detecting both or one of the pair of left and right shoulder lines based on the markers in the image sent from the vehicle-mounted control device, and calculating the roadway area based on the thus detected road shoulder lines, means for recognizing the running state of the own vehicle on the running surface provided by the running path building tool;
Means for generating a command indicating a correction amount for driving and steering necessary to bring the recognized driving state of the own vehicle to a desired driving state, and transmitting the command to the vehicle-mounted control device;
A computer program to act as

Images