WO2018077308A1

WO2018077308A1 - Electronic device control method, electronic device and computer storage medium

Info

Publication number: WO2018077308A1
Application number: PCT/CN2017/110859
Authority: WO
Inventors: 卿明
Original assignee: 纳恩博（北京）科技有限公司
Priority date: 2016-10-26
Filing date: 2017-11-14
Publication date: 2018-05-03
Also published as: CN106647743A

Abstract

An electronic device control method, an electronic device and a computer storage medium. The method comprises: obtaining image data, performing road feature analysis on the image data, and obtaining a road image (S101); performing road type identification on the road image, and obtaining corresponding road type information (S102); according to the road type information, and on the basis of a preset control strategy, generating a control command, so as to execute the control command (S103).

Description

Control method of electronic device, electronic device and computer storage medium

Cross-reference to related applications

The application is based on a Chinese patent application with the application number of 201611024939.7, the application date being November 14, 2016, and the Chinese patent application with the application number of 201610949963.5 and the application date being October 26, 2016, and the Chinese patent application is required. The entire contents of this Chinese patent application are hereby incorporated herein by reference.

Technical field

The present invention relates to the field of artificial intelligence technologies, and in particular, to a method for controlling an electronic device, an electronic device, and a computer storage medium.

Background technique

With the increase of labor costs and the development of sensors and other technologies, the use of artificial intelligence devices such as robots or smart balance vehicles has increased day by day. The driving function is one of the most important basic functions of artificial intelligence devices.

However, in the driving process of the manned, remote control or tracking mode, the current artificial intelligence device mainly travels according to the on-site operation of the control personnel or according to the pre-route setting, and is prone to slippage when encountering special circumstances such as rain, snow, blind roads and the like. Sudden dangers such as violent shaking, causing casualties or equipment damage.

Summary of the invention

The embodiments of the present invention are expected to provide a control method for an electronic device, an electronic device, and a computer storage medium. The technical problem that the artificial intelligence device has low security when encountering a special road surface environment during driving is solved.

In a first aspect, an embodiment of the present invention provides a method for controlling an electronic device, including: obtaining image data, performing road feature analysis on the image data, obtaining a road image; performing road type recognition on the road image, and obtaining corresponding The road type information; based on the road type information, generating a control instruction to execute the control instruction based on the preset control policy.

In an embodiment, performing road feature analysis on the image data to obtain a road image includes: analyzing, according to preset road feature information, an image matching the road feature information from the image data The area is represented by the image area as the road image.

In an embodiment, the road type identification is performed on the road image, and the corresponding road type information is obtained, including: extracting feature information in the road image according to a preset extraction rule; and based on the feature information and the pre- Corresponding relationship between the road feature and the road type is determined, and road type information corresponding to the road image is determined.

In an embodiment, the road type identification is performed on the road image, and the corresponding road type information is obtained, including: extracting feature information in the road image according to a preset extraction rule; and based on a preset machine learning model Determining the road type information corresponding to the feature information; wherein, based on the comparison result of the road type information and the road type correction information input by the user, a correction instruction is generated to correct the machine learning model based on the correction instruction.

In an embodiment, the generating, according to the road type information, a control instruction based on a preset control policy, including a combination of one or more of the following: generating a deceleration based on characterizing the road type as the first road type information Controlling instructions to control a speed of movement of the electronic device to be less than a preset speed threshold; or generating a steering control command to control electronic equipment steering based on characterizing the road type as second road type information; or based on characterizing the road type The three road type information generates an alarm instruction to control the electronic device to output an alarm signal to prompt the user; or generates a torque adjustment command based on the road type characterizing the road type to control the torque output of the electronic device.

In an embodiment, the method further includes: detecting that the electronic device is executing the control instruction a driving state, obtaining a driving state parameter; determining whether the driving state parameter satisfies a preset requirement; if the driving state parameter satisfies a preset requirement, controlling the electronic device to maintain the driving state; if the driving state If the parameter does not satisfy the preset requirement, an adjustment command is generated based on the driving state parameter to execute the adjustment command to adjust the driving state.

In an embodiment, before the generating the control instruction, the method further includes: detecting a current operation mode of the electronic device, and obtaining current mode information; and generating, according to the road type information, a control instruction according to the preset control policy. The method includes: when the current mode information indicates that the electronic device is in a riding state, generating a first control instruction according to the preset riding control policy according to the obtained road type information; and when the current mode information indicates When the electronic device is in the non-riding state, according to the obtained road type information, a second control instruction is generated based on the preset non-riding control policy.

In a second aspect, an embodiment of the present invention further provides an electronic device, including:

An image acquisition module configured to obtain image data;

a road extraction module configured to perform road feature analysis on the image data obtained by the image acquisition module to obtain a road image;

a type identification module configured to perform road type identification on the road image obtained by the road extraction module, and obtain corresponding road type information;

And a control module configured to generate, according to the preset control policy, the control instruction according to the road type information obtained by the type identification module to execute the control instruction.

In an embodiment, the road extraction module is configured to, based on the preset road feature information, analyze and obtain an image region that matches the road feature information from the image data, where the image region is used as the image region Road image.

In an embodiment, the type identifying module further includes: a first extracting unit configured to extract feature information in the road image according to a preset extraction rule; and a first determining unit configured to be based on the first extracting The feature information extracted by the unit and the preset road features and road types Corresponding relationship determines road type information corresponding to the road image.

In an embodiment, the type identification module further includes: a second extracting unit configured to extract feature information in the road image according to a preset extraction rule; and a second determining unit configured to perform machine learning based on preset a model, determining road type information corresponding to the feature information extracted by the second extracting unit; wherein, based on the comparison result of the road type information and the road type correction information input by the user, generating a correction instruction to The correction instruction corrects the machine learning model.

In an embodiment, the control module is configured to generate a deceleration control command to control the movement speed of the electronic device to be less than a preset speed threshold based on the road type characterizing the road type as the first road type information; or based on characterizing the road Type is second road type information, generating a steering control command to control electronic equipment steering; or generating an alarm command based on characterizing the road type as third road type information, to control the electronic device to output an alarm signal to prompt the user; or based on the representation The road type is fourth road type information, and a torque adjustment command is generated to control the torque output of the electronic device.

In an embodiment, the electronic device further includes: a detecting module configured to detect a driving state after the execution of the control command, and acquire a driving state parameter; and an adjustment module configured to determine the driving acquired by the detecting module Whether the state parameter meets the preset requirement; if the driving state parameter meets the preset requirement, the driving state is maintained; if the driving state parameter does not satisfy the preset requirement, generating an adjustment instruction based on the driving state parameter, The adjustment command is executed to adjust the driving state.

In an embodiment, the electronic device further includes: a mode module configured to detect an operation mode to obtain current mode information; the control module includes: a riding unit configured to be the current mode obtained by the mode module The information indicates that when the electronic device is in the riding state, according to the obtained road type information, a first control instruction is generated based on the preset riding control policy; the non-riding unit is configured to be obtained by the mode module. The current mode information indicates the electronic When the device is in the non-riding state, according to the obtained road type information, a second control instruction is generated based on the preset non-riding control strategy.

In a third aspect, an embodiment of the present invention further provides an electronic device, including: a processor and a memory for storing a computer program capable of running on a processor, wherein when the processor is configured to run the computer program, The steps of the control method of the electronic device according to the embodiment of the invention are performed.

In a fourth aspect, the embodiment of the present invention further provides a computer storage medium, where the computer storage medium stores computer executable instructions, and the computer executable instructions are used to perform control of the electronic device according to the embodiment of the present invention. method.

The control method of the electronic device, the electronic device and the computer storage medium provided by the embodiments of the present application collect the road image of the current traveling road during the running, and analyze the image in real time to identify the current road type information, and according to the road The type information is used to control the driving state of the electronic device in a targeted manner, so that the electronic device can execute different control commands according to different road conditions, and adopt different driving states in a targeted manner to effectively improve the safety of the whole driving process.

The above description is only an overview of the technical solutions of the present invention, and the above-described and other objects, features and advantages of the present invention can be more clearly understood. Specific embodiments of the invention are set forth below.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are merely embodiments of the present invention, Those of ordinary skill in the art will be able to obtain other figures from the drawings provided without the inventive effort.

1 is a flowchart of a method for controlling an electronic device according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The embodiment of the present application solves the technical problem that the artificial intelligence device in the prior art has low security when encountering a special road environment during driving by providing a control method for an electronic device, an electronic device, and a computer storage medium. . The technical effect of improving the safety of driving throughout the journey is achieved.

To solve the above technical problem, the general idea of the technical solution provided by the embodiment of the present application is as follows:

When the electronic device is traveling, performing road feature analysis on the image data acquired by the image acquisition module of the electronic device to obtain a road image; performing road type recognition on the road image to obtain corresponding road type information; And generating, according to the preset control policy, the road control information, so that the electronic device executes the control instruction.

The solution provided by the present application collects the road image of the current traveling road during the running, analyzes the image in real time to identify the current road type information, and controls the driving state of the electronic device according to the road type information. The electronic device can execute different control commands according to different road conditions, and adopt different driving states to effectively improve the safety of the whole driving process.

The technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. It is a partial embodiment of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

Embodiment 1

In this embodiment, a method for controlling an electronic device is provided. Referring to FIG. 1, as shown in FIG. 1, the method includes:

Step S101, obtaining image data, performing road feature analysis on the image data, and obtaining Road image.

Step S102, performing road type identification on the road image, and obtaining corresponding road type information.

Step S103, generating a control instruction to execute the control instruction based on the road type information based on the preset control policy.

The method can be applied to an electronic device having a driving or flying function. For example, the electronic device is, for example, a robot, a balance car or an aircraft, and is not enumerated here.

Hereinafter, the control method of the electronic device provided by the present application is described in detail in conjunction with FIG. 1 :

In this embodiment, step S101 is performed to obtain image data, and road feature analysis is performed on the image data to obtain a road image. Wherein, as an implementation manner, an image acquisition module is disposed in the electronic device, and the image acquisition module of the electronic device collects image data. In another embodiment, the electronic device is coupled to another electronic device having an image acquisition module that obtains image data acquired by the other electronic device.

In an embodiment, the image acquisition module may be at least one image acquisition component of a depth camera, a vision sensor, an infrared image acquisition module, and a laser image acquisition module, which is not limited herein; the number of the image acquisition modules may be For one or more, there is no limit here.

It should be noted that the image capturing module may perform image capturing in real time during the movement of the electronic device, or may perform image capturing intermittently according to a preset time interval, or may be a satellite panoramic image obtained in advance. Image acquisition is performed in a predetermined geographical location, and is not limited in this application.

In the specific implementation process, the road feature analysis is performed on the image data, and the method for obtaining the road image may be various. The following two examples are as follows:

The first type, matching road features, obtains road images. That is, performing the road feature analysis on the image data to obtain the road image includes: analyzing, according to the preset road feature information, an image region that is matched with the road feature information from the image data, Image The area serves as the road image.

Specifically, the road feature information may be preset to be stored in the electronic device or stored in the cloud server, and the road feature information may include at least one of the following road features: a straight line, a blind line, a zebra line, a turning arrow, a roadside ladder The geometric features may also include at least one of the following features: green color of the grass, stone gray of the road, white color of the snow, and the like, which are not enumerated here. Then, an image matching algorithm or an image extraction algorithm is used to analyze whether there is an image feature matching the preset road feature information in the image data, and if so, the region where the image feature is located is used as the road image.

Second, exclude non-road features and obtain road images.

Specifically, the non-road feature information may be preset to be stored in the electronic device or stored in the cloud server, and the non-road feature information may include geometric features such as a tree line, a pedestrian line, a step, a house line, and a street light line. Including: the color of the street light, the flashing of the traffic lights, etc., are not listed here. And using an image matching algorithm or an image extraction algorithm to analyze whether there is an image feature matching the preset non-road feature information in the image data, and if present, excluding the region where the image feature is located to exclude the remaining The image area serves as the road image.

Of course, in the specific implementation process, the road image is not limited to the above methods, and different acquisition methods may be determined based on different requirements, and the embodiments of the present invention are not listed in detail and are not limited.

In this embodiment, step S102 is performed to perform road type identification on the road image, and obtain corresponding road type information.

In the specific implementation process, the road type identification is performed on the road image, and the method for obtaining the corresponding road type information may be various. The following two examples are as follows:

The first one uses feature analysis to identify road types. The method based on feature analysis mainly extracts texture features, edge features, line features, etc. of the pavement to form feature vectors. The trained classifier or some conditional rules are used to classify and judge the input feature vector to complete the recognition of the road. That is, the road type identification is performed on the road image, and the corresponding road type information is obtained, including: extracting feature information in the road image according to a preset extraction rule; and based on the feature information and the preset road feature Corresponding relationship of the road type, the road type information corresponding to the road image is determined.

Specifically, the correspondence between the road feature and the road type may be stored in the electronic device or the cloud server in advance. For example, the blind track type corresponds to a road feature with short parallel lines and slightly protruding lines; the zebra crossing corresponds to white parallel lines and lines. Plane road features; snow corresponds to large white road features; grass corresponds to road features with green and grass contour lines; water ripple corresponds to high-reflective road features; pitted road corresponds to concave road features; The road features corresponding to the convex lines and the yellow and black colors are not listed here.

And performing the targeted extraction on the road image according to the road feature; determining the extracted road feature that matches the feature information, and determining the road type information according to the road type corresponding to the matched road feature. For example, if green feature information is extracted in the road image, it may be determined that the road type corresponding to the road image is a grassland according to a pre-existing grassland corresponding to a road feature having green and grass outline lines; The feature information of the short parallel lines and the slightly convex lines is extracted from the image, and the road features corresponding to the road image are determined to be blind roads according to the pre-existing blind road type corresponding to the road features with short parallel lines and slightly convex lines.

Second, the neural network method is used to identify the road type. The neural network-based method needs to construct a large training sample, directly input the sample into the neural network for training, and obtain a trained model, and use the model to complete the recognition of a specific target. That is, the road type identification is performed on the road image, and the corresponding road type information is obtained, including: extracting feature information in the road image according to a preset extraction rule; and determining, according to a preset machine learning model, Road type information corresponding to the feature information; wherein, based on the road type information and user input The alignment result of the road type correction information generates a correction command to enable the electronic device to correct the machine learning model based on the correction instruction.

Specifically, a large number of training samples are stored in advance in the electronic device or the cloud server, that is, a large number of road image samples are stored, and corresponding road types are determined in advance for each road image; the neural network pairs a large number of road images and corresponding image type samples. Training is performed to obtain the machine learning model, wherein the machine learning model takes the feature information as an input parameter and the road type as an output result.

When it is necessary to identify the road type, the electronic device extracts feature information such as color, line, unevenness, and shininess in the road image, and then brings the feature information into the machine learning model obtained in advance, thereby calculating the road. Type information.

After each calculation is completed, a correction instruction may be generated based on the comparison result of the road type information and the road type correction information input by the user to correct the machine learning model based on the correction instruction.

Certainly, in the specific implementation process, the road type information is not limited to the above methods, and different determination methods may be determined based on different requirements, and the embodiments of the present invention are not enumerated in detail, and are not limited.

Step S103 is executed, and according to the road type information, a control instruction is generated based on the preset control policy to execute the control instruction.

The method does not limit the identified road type and the corresponding control command, and the identified road type may be pre-defined, and the corresponding control command may be to issue a signal prompt or automatically change the state of the electronic device itself or transfer the state to the electronic device. Other modules, below are several methods for generating control commands based on the road type information, including:

Deducing a deceleration control command to control a movement speed of the electronic device to be less than a preset speed threshold based on the road type indicating that the road type is first road type information; the first road type information includes at least one of the following road types: water Pit, potholes, sand, speed bumps, sidewalks, Snow, etc.; or

Generating a steering control command to control electronic device steering based on characterizing the road type as second road type information; the second road type information includes at least one of the following road types: obstacle ground, puddle, pothole, etc. ;or

And generating an alarm instruction to control the electronic device to output an alarm signal to prompt the user based on the road type indicating that the road type is the third road type information; the third road type information includes at least one of the following road types: an obstacle ground, a pothole, etc. .

For example, taking the electronic device as a robot, for example, when it is recognized as an ice or snow road condition or a water storage road condition, the robot will limit the maximum traveling speed to ensure the safety of the user; when identified as a blind road, the robot Send out a signal to remind the user to drive away from the blind road or reduce the speed to ensure the safety of the person or the disabled. This function can also be specially set to set the robot to drive only on the blind road to provide guidance for the visually impaired; When the speed bump is reached, the robot sends out a signal to prompt the user to decelerate or automatically slow down the speed; when other obstacles are recognized, a signal prompt or automatic deceleration stops.

In a specific implementation process, the torque adjustment command may be generated to control the torque output of the electronic device based on the road type characterizing the road type as the fourth road type information.

Specifically, considering that when driving on different types of roads, there will be different jitter conditions. In order to ensure that electronic devices such as robots can stably travel on different types of roads, the torque output of the devices can be adjusted according to different road types. When the torque output is larger, the attitude control of the electronic device such as a robot is more stable, and it is not easy to fall. For example, when the road type is snow, in order to prevent the electronic device from tilting, an enhanced torque command can be generated to enhance the output torque and improve the running stability of the device; when the road type becomes a cement road surface, the reduction can be generated. The torque command reduces the torque of the output and saves energy while maintaining the running stability of the equipment.

In order to ensure that the driving state of the electronic device after the execution of the control command can meet the requirement or be in a safe state, the control command may be set to execute the control command, The method further includes: detecting a driving state of the electronic device after executing the control instruction, acquiring a driving state parameter; determining whether the driving state parameter meets a preset requirement; and if the driving state parameter satisfies a preset requirement, controlling the The electronic device maintains the driving state; if the driving state parameter does not satisfy the preset requirement, generating an adjustment command based on the driving state parameter to execute the adjustment command to adjust the driving state.

For example, after the electronic device executes the control instruction, it may detect that the driving state parameter of the electronic device is obtained, and the driving state parameter includes at least one of the following parameters: driving speed, jitter frequency and amplitude, and motion. Acceleration, degree of deflection.

When the driving speed exceeds the preset speed requirement, generate a deceleration adjustment command to adjust the driving speed to the safe range; or, when the jitter frequency and amplitude exceed the preset range requirement, it is considered that there is a danger of equipment damage and a risk of equipment turnover, and a deceleration adjustment is generated. Commanding to adjust the driving speed to control the jitter in a safe range; or, when the motion acceleration exceeds the preset range requirement, it is considered that there is a risk of equipment turnover, and an acceleration adjustment command is generated to adjust the driving acceleration to control the driving in a safe range; Or, when the degree of skewness of the electronic device exceeds the preset range, an alarm, deceleration or stop adjustment command is generated to prevent the device from rolling over.

Of course, in the specific implementation process, more driving state parameters can also be detected to ensure that the driving state of the electronic device is in a safe range, which will not be enumerated here.

Considering that an electronic device can have multiple operating modes, different control modes may be generated for different road types in different operating modes, and thus may be set before the generating the control command, the method further comprising: Detecting a current mode of operation of the electronic device to obtain current mode information; and generating, according to the road type information, a control instruction according to the preset control policy, including: when the current mode information indicates that the electronic device is riding In the row state, according to the obtained road type information, generating a first control instruction based on the preset riding control policy; and when the current mode information indicates that the electronic device is in a non-riding state, according to the obtained The road type information generates a second control command based on the preset non-riding control strategy.

Specifically, the common operation modes of the electronic device as the balance car include a riding mode, a tracking mode, and a remote control mode, and the tracking mode and the remote control mode may be set to be non-riding mode. Among them, the robot is in the trailing state of the target while in the tracking mode. In the riding mode, it is mainly necessary to consider the riding comfort and safety of the person. In the non-riding mode, the protection of the device is mainly considered, so in different operating modes, even the same road type, the corresponding electronic device generates Control instructions can also be different. E.g:

When determining that the road type is pitted ground, if the current mode information indicates that the electronic device is in a riding state, in order to ensure riding comfort, a first deceleration command to decelerate to a first speed may be generated; if current mode information In order to indicate that the electronic device is in a non-riding state, in order to protect the device, a second deceleration command to decelerate to a second speed may be generated, wherein the first speed is less than the second speed.

When determining that the road type is a blind road, if the current mode information indicates that the electronic device is in a riding state, an alarm instruction is generated to alert the user to determine whether to travel in a blind lane according to further operations of the user; if current mode information indicates that If the electronic device is in a non-riding state, in order to avoid disturbing the disabled, a steering command can be generated to bypass the blind road.

When determining that the road type is snow, if the current mode information indicates that the electronic device is in a riding state, a deceleration command is generated for user safety to prevent an accident from slipping; if the current mode information indicates that the electronic device is not In the riding state, energy-saving instructions can be generated to reduce the energy output of the power supply unit, so that the device can directly use the low-friction sliding of the snow to achieve energy-saving effects.

Of course, in the specific implementation process, the control commands generated for different operation modes can be set as needed, and are not limited to the above, and are not enumerated here.

Embodiment 2

This embodiment provides an electronic device. Referring to FIG. 2, the electronic device includes:

The image acquisition module 201 is configured to obtain image data;

The road extraction module 202 is configured to perform road feature analysis on the image data obtained by the image acquisition module 201 to obtain a road image;

The type identification module 203 is configured to perform road type identification on the road image obtained by the road extraction module 202, and obtain corresponding road type information;

The control module 204 is configured to generate a control instruction to execute the control instruction based on the road type information obtained by the type identification module 203 based on the preset control policy.

The electronic device is specifically an electronic device having a driving or flying function. For example, the electronic device is, for example, a robot, a balance car or an aircraft, and is not listed here.

In the embodiment of the present application, the road extraction module 202 is configured to analyze and obtain an image region that matches the road feature information from the image data based on the preset road feature information, and use the image region as the image region. The road image.

As an implementation manner, the type identification module 203 further includes:

a first extracting unit configured to extract feature information in the road image according to a preset extraction rule;

The first determining unit is configured to determine road type information corresponding to the road image based on the feature information extracted by the first extracting unit and the corresponding relationship between the preset road feature and the road type.

As another implementation manner, the type identification module 203 further includes:

a second extracting unit configured to extract feature information in the road image according to a preset extraction rule;

a second determining unit, configured to determine road type information corresponding to the road image extracted by the second extracting unit, based on a preset machine learning model;

Wherein, based on the comparison result of the road type information and the road type correction information input by the user, a correction instruction is generated to correct the machine learning model based on the correction instruction.

In an embodiment, the control module 204 is configured to represent the road type as a road type information, generating a deceleration control command to control the movement speed of the electronic device to be less than a preset speed threshold; or

Generating a steering control command to control electronic device steering based on characterizing the road type as second road type information; or

Generating an alarm instruction to control the electronic device to output an alarm signal to prompt the user based on characterizing the road type as the third road type information; or

Based on characterizing the road type as the fourth road type information, a torque adjustment command is generated to control the torque output of the electronic device.

In the embodiment of the present application, the electronic device may further include:

a detecting module configured to detect a driving state after executing the control command, and acquire a driving state parameter;

An adjustment module, configured to determine whether the driving state parameter acquired by the detecting module meets a preset requirement; if the driving state parameter meets a preset requirement, controlling the electronic device to maintain the driving state; If the state parameter does not satisfy the preset requirement, an adjustment command is generated based on the driving state parameter to execute the adjustment command to adjust the driving state.

In an embodiment, the electronic device may further include: a mode module configured to detect an operation mode and obtain current mode information;

The control module 204 includes:

a riding unit configured to generate a first control based on the preset riding control policy according to the obtained road type information when the current mode information obtained by the mode module indicates that the electronic device is in a riding state instruction;

a non-riding unit configured to: when the current mode information obtained by the mode module indicates that the electronic device is in a non-riding state, generate, according to the obtained road type information, based on a preset non-riding control policy The second control command.

An electronic device according to an embodiment of the present invention is an electronic device that implements the first embodiment of the present invention. The electronic device used in the control method, and therefore, based on the method described in the first embodiment of the present invention, those skilled in the art can understand the specific structure and deformation of the device, and thus will not be described herein. The apparatus used in the method of the first embodiment of the present invention is within the scope of the present invention.

In this embodiment, the road extraction module 202, the type identification module 203, and the control module 204 in the electronic device may be implemented by a central processing unit (CPU), a digital signal processor (DSP, Digital). The signal processor, the Micro Control Unit (MCU) or the Field-Programmable Gate Array (FPGA) is implemented; the image acquisition module 201 in the electronic device can be implemented by a camera in practical applications.

It should be noted that, when the electronic device provided by the foregoing embodiment performs control processing, only the division of each of the foregoing program modules is illustrated. In actual applications, the processing may be allocated by different program modules as needed. The internal structure of the electronic device is divided into different program modules to perform all or part of the processing described above. In addition, the embodiment of the electronic device and the control method of the electronic device provided by the foregoing embodiments are the same concept, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.

The method and the electronic device provided by the embodiments of the present application collect the road image of the current traveling road during the running, analyze the image in real time to identify the current road type information, and specifically control the electronic according to the road type information. The driving state of the device enables the electronic device to execute different control commands according to different road conditions, and adopts different driving states in a targeted manner to effectively improve the safety of the whole driving process.

Embodiments of the present invention also provide an electronic device including: a processor and a memory for storing a computer program executable on the processor. among them,

The memory can be implemented by any type of volatile or non-volatile storage device, or a combination thereof. The non-volatile memory may be a read only memory (ROM), a programmable read only memory (PROM, Programmable Read-Only Memory), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Magnetic Random Access Memory (FRAM, Ferromagnetic Random) Access Memory), Flash Memory, Magnetic Surface Memory, Optical Disk, or Compact Disc Read-Only Memory (CD-ROM); magnetic surface memory can be disk storage or tape storage. The volatile memory can be a random access memory (RAM) that acts as an external cache. By way of example and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access (SSRAM). DRAM (Dynamic Random Access Memory), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), enhancement Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Dynamic Random Access Memory (SLDRAM), Direct Memory Bus Random Access Memory (DRRAM) ). The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

The method disclosed in the foregoing embodiments of the present invention may be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method may be completed by an integrated logic circuit of hardware in a processor or an instruction in a form of software. The above described processor may be a general purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or the like. The processor may implement or perform the methods, steps, and logic blocks disclosed in the embodiments of the present invention. General processing The device can be a microprocessor or any conventional processor or the like. The steps of the method disclosed in the embodiment of the present invention may be directly implemented as a hardware decoding processor, or may be performed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, the storage medium being located in the memory, the processor reading the information in the memory, and completing the steps of the foregoing methods in combination with the hardware thereof.

In the embodiment of the present invention, when the processor is used to run the computer program, performing: obtaining image data, performing road feature analysis on the image data, obtaining a road image; performing road type identification on the road image, and obtaining corresponding The road type information; based on the road type information, generating a control instruction to execute the control instruction based on the preset control policy.

In an embodiment, when the processor is configured to run the computer program, performing: analyzing, according to preset road feature information, an image region that matches the road feature information from the image data, The image area is described as the road image.

In an embodiment, when the processor is configured to run the computer program, performing: extracting feature information in the road image according to a preset extraction rule; and based on the feature information and preset road features and road types Corresponding relationship determines road type information corresponding to the road image.

In an embodiment, when the processor is configured to run the computer program, performing: extracting feature information in the road image according to a preset extraction rule; and determining, according to a preset machine learning model, the feature information The road type information; wherein, based on the comparison result of the road type information and the road type correction information input by the user, a correction instruction is generated to correct the machine learning model based on the correction instruction.

In an embodiment, when the processor is configured to run the computer program, executing: generating, according to the road type information, a control instruction based on a preset control policy, including a combination of one or more of the following: The road type is the first road type information, and generates a deceleration control command to control the movement speed of the electronic device to be less than a preset speed threshold; or based on characterizing the road The road type is second road type information, generating a steering control command to control electronic equipment steering; or generating an alarm command based on characterizing the road type as third road type information, to control the electronic device to output an alarm signal to prompt the user; or based on Characterizing the road type as fourth road type information, generating a torque adjustment command to control the torque output of the electronic device.

In an embodiment, the processor is configured to: when detecting the computer program, perform: detecting a driving state of the electronic device after executing the control instruction, acquiring a driving state parameter; and determining whether the driving state parameter meets a preset requirement And if the driving state parameter meets a preset requirement, controlling the electronic device to maintain the driving state; if the driving state parameter does not satisfy the preset requirement, generating an adjustment instruction based on the driving state parameter to execute the The adjustment command is described to adjust the driving state.

In an embodiment, when the processor is configured to run the computer program, executing: detecting a current operating mode of the electronic device to obtain current mode information before generating the control command; and when the current mode information indicates that the electronic device is In the riding state, based on the obtained road type information, generating a first control instruction based on the preset riding control policy; and when the current mode information indicates that the electronic device is in a non-riding state, according to the obtained The road type information is generated based on a preset non-riding control strategy to generate a second control instruction.

In an exemplary embodiment, an embodiment of the present invention further provides a computer readable storage medium, such as a memory including a computer program, which may be executed by a processor of an electronic device to perform the steps of the foregoing method. The computer readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories, such as Mobile phones, computers, tablet devices, personal digital assistants, etc.

An embodiment of the present invention provides a computer storage medium, where the computer storage medium stores computer executable instructions, where the computer executable instructions are used to: obtain image data, perform road feature analysis on the image data, and obtain Road image; performing the road image The road type is identified, and corresponding road type information is obtained; and according to the road type information, a control instruction is generated based on the preset control policy to execute the control instruction.

In an embodiment, the computer executable instructions are configured to: obtain an image region matching the road feature information from the image data based on preset road feature information, and use the image region as The road image.

In an embodiment, the computer executable instructions are configured to: extract feature information in the road image according to a preset extraction rule; and determine, according to the feature information and a corresponding relationship between a preset road feature and a road type, The road type information corresponding to the road image.

In an embodiment, the computer executable instructions are configured to: extract feature information in the road image according to a preset extraction rule; and determine road type information corresponding to the feature information based on a preset machine learning model Wherein, based on the comparison result of the road type information and the road type correction information input by the user, a correction instruction is generated to correct the machine learning model based on the correction instruction.

In an embodiment, the computer executable instructions are configured to: generate, according to the road type information, a control instruction based on a preset control policy, including a combination of one or more of the following: based on characterizing the road type a first road type information, generating a deceleration control command to control a movement speed of the electronic device to be less than a preset speed threshold; or generating a steering control command to control the electronic device steering based on characterizing the road type as the second road type information; or And generating an alarm instruction to control the electronic device to output an alarm signal to prompt the user based on the road type indicating that the road type is the third road type information; or generating a torque adjustment instruction to control the electronic device based on the road type characterizing the road type as the fourth road type information Torque output.

In an embodiment, the computer executable instructions are configured to: detect a driving state of the electronic device after executing the control instruction, acquire a driving state parameter; determine whether the driving state parameter satisfies a preset requirement; If the driving state parameter meets the preset requirement, the electronic device is controlled to maintain the driving state; if the driving state parameter does not meet the preset requirement, then An adjustment command is generated based on the driving state parameter to execute the adjustment command to adjust a driving state.

In an embodiment, the computer executable instructions are configured to: detect a current operating mode of the electronic device to obtain current mode information before generating the control command; and when the current mode information indicates that the electronic device is in a riding state And generating, according to the obtained road type information, a first control instruction according to the preset riding control policy; and when the current mode information indicates that the electronic device is in a non-riding state, according to the obtained road type information And generating a second control instruction based on the preset non-riding control strategy.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, an electronic device, or a computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of a method, an electronic device, and a computer program product according to an embodiment of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Means for implementing the functions specified in one or more of the flow or in a block or blocks of the flow chart.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The device is implemented in a flow chart A function specified in a block or blocks of a process or multiple processes and/or block diagrams.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

While the preferred embodiment of the invention has been described, it will be understood that Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and the modifications and

It is apparent that those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the embodiments of the invention. Thus, it is intended that the present invention cover the modifications and modifications of the embodiments of the invention.

Industrial applicability

The technical solution of the embodiment of the present invention collects the road image of the current traveling road during driving, analyzes the image in real time to identify the current road type information, and controls the driving state of the electronic device according to the road type information. The electronic device can execute different control commands according to different road conditions, and adopt different driving states to effectively improve the safety of the whole driving process.

Claims

A method of controlling an electronic device, the method comprising:

Obtaining image data, performing road feature analysis on the image data, and obtaining a road image;

Performing road type identification on the road image to obtain corresponding road type information;

And generating, according to the road type information, a control instruction to execute the control instruction based on the preset control policy.
The method of claim 1 wherein said performing road feature analysis on said image data to obtain a road image comprises:

Based on the preset road feature information, an image region matching the road feature information is obtained from the image data, and the image region is used as the road image.
The method of claim 1, wherein the performing the road type identification on the road image and obtaining the corresponding road type information comprises:

Extracting feature information in the road image according to a preset extraction rule;

And determining road type information corresponding to the road image based on the feature information and the correspondence between the preset road feature and the road type.
The method of claim 1, wherein the performing the road type identification on the road image and obtaining the corresponding road type information comprises:

Extracting feature information in the road image according to a preset extraction rule;

Determining road type information corresponding to the feature information based on a preset machine learning model;

Wherein, based on the comparison result of the road type information and the road type correction information input by the user, a correction instruction is generated to correct the machine learning model based on the correction instruction.
The method of claim 1, wherein the generating a control instruction based on the road type information based on a preset control policy comprises one or more of the following combinations:

Generating a deceleration control command to control the movement speed of the electronic device to be less than a preset speed threshold based on characterizing the road type as the first road type information; or

Generating a steering control command to control electronic device steering based on characterizing the road type as second road type information; or

Generating an alarm instruction to control the electronic device to output an alarm signal to prompt the user based on characterizing the road type as the third road type information; or

Based on characterizing the road type as the fourth road type information, a torque adjustment command is generated to control the torque output of the electronic device.
The method of any of claims 1 to 5, wherein the method further comprises:

Detecting a driving state of the electronic device after executing the control instruction, and acquiring a driving state parameter;

Determining whether the driving state parameter meets a preset requirement;

Controlling the electronic device to maintain the driving state if the driving state parameter meets a preset requirement;

If the driving state parameter does not satisfy the preset requirement, an adjustment command is generated based on the driving state parameter to execute the adjustment command to adjust the driving state.
The method according to any one of claims 1 to 5, wherein before the generating the control instruction, the method further comprises: detecting a current operating mode of the electronic device, and obtaining current mode information;

And generating, according to the road type information, a control instruction based on the preset control policy, including:

And when the current mode information indicates that the electronic device is in a riding state, generating a first control instruction according to the preset riding control policy according to the obtained road type information;

And when the current mode information indicates that the electronic device is in a non-riding state, generating a second control instruction according to the preset non-riding control policy according to the obtained road type information.
An electronic device comprising:

An image acquisition module configured to obtain image data;

a road extraction module configured to perform road feature analysis on the image data obtained by the image acquisition module to obtain a road image;

a type identification module configured to perform road type identification on the road image obtained by the road extraction module, and obtain corresponding road type information;

And a control module configured to generate, according to the preset control policy, the control instruction according to the road type information obtained by the type identification module to execute the control instruction.
The electronic device according to claim 8, wherein the road extraction module is configured to analyze and obtain an image region matching the road feature information from the image data based on preset road feature information. The image area is described as the road image.
The electronic device of claim 8, wherein the type identification module further comprises:

a first extracting unit configured to extract feature information in the road image according to a preset extraction rule;

The first determining unit is configured to determine road type information corresponding to the road image based on the feature information extracted by the first extracting unit and the corresponding relationship between the preset road feature and the road type.
The electronic device of claim 8, wherein the type identification module further comprises:

a second extracting unit configured to extract feature information in the road image according to a preset extraction rule;

a second determining unit, configured to determine road type information corresponding to the feature information extracted by the second extracting unit, based on a preset machine learning model;

Wherein, based on the comparison result of the road type information and the road type correction information input by the user, a correction instruction is generated to correct the machine learning model based on the correction instruction.
The electronic device of claim 8, wherein the control module is configured to generate a deceleration control command based on the road type characterizing the road type to control the movement speed of the electronic device to be less than a preset speed threshold; or

Generating a steering control command to control electronic device steering based on characterizing the road type as second road type information; or

Generating an alarm instruction to control the electronic device to output an alarm signal to prompt the user based on characterizing the road type as the third road type information; or

Based on characterizing the road type as the fourth road type information, a torque adjustment command is generated to control the torque output of the electronic device.
The electronic device according to any one of claims 8 to 12, further comprising:

a detecting module configured to detect a driving state after executing the control command, and acquire a driving state parameter;

The adjustment module is configured to determine whether the driving state parameter acquired by the detecting module satisfies a preset requirement; if the driving state parameter meets a preset requirement, maintaining the driving state; if the driving state parameter does not satisfy the preset If the request is made, an adjustment command is generated based on the driving state parameter to execute the adjustment command to adjust the driving state.
An electronic device according to any one of claims 8 to 12, wherein:

The electronic device further includes: a mode module configured to detect an operation mode and obtain current mode information;

The control module includes:

a riding unit configured to generate a first control based on the preset riding control policy according to the obtained road type information when the current mode information obtained by the mode module indicates that the electronic device is in a riding state instruction;

a non-riding unit configured to: when the current mode information obtained by the mode module indicates that the electronic device is in a non-riding state, generate, according to the obtained road type information, based on a preset non-riding control policy The second control command.
An electronic device comprising: a processor and a memory for storing a computer program executable on a processor, wherein the processor is configured to execute the computer program, perform any one of claims 1 to 7 The steps of the method of controlling the electronic device.
A computer storage medium storing computer executable in the computer storage medium The computer-executable instructions for performing the control method of the electronic device according to any one of claims 1 to 7.