The research of Intelligent Autonomous Vehicles (IAV) is related to computer measurement and control, computer vision, sensor technologies, vehicle engineering and many other fields. In general, it can be said that the research of IAV is to integrate computer vision and computer control with vehicle engineering.
At the beginning of the research on highway automation, it was planned to make use of buried cables for intelligent vehicle navigation, which was based on electromagnetic induction. However, the range of measurable electromagnetic field is rather narrow (decimeter level). Besides this method cannot provide needed information, location information and obstacles information, for vehicles. As a result, this method for navigation has been abandoned in the study of IAV.
In the middle of 1990s, the idea of autonomous road vehicle guidance by magnetic nails implanted into the lane center has been presented in the United States and Japan . Experiments have been carried out on the highway in Japan and the United States in 1996 and 1997 respectively. Nevertheless, cost for this infrastructure would be so expensive that there would be limited value to take this method.
In fact, the information received is mostly from people's vision when people driving. Traffic signals, Traffic patterns, road signs can be regarded as visual communication language, which provides guidance for drivers. Obviously, it is natural to take into account the application of computer vision to interpret the environmental language.
An intelligent vehicle system which is of great value must have following characteristics: real-time, robustness and practicability. The first one, real-time, means that the data processing of the system must be synchronized with the speed of vehicles. The second one refers to the adaptability of the intelligent vehicles. There are many different types of roads, such as highways, urban roads, the general roads. And the conditions of these roads, such as the width of pavement and lane, color, texture, dynamic random obstacles and traffic flow, differ greatly. Changing climate conditions, such as sunshine and shadows, dusk and night, rain and snow, also affect the performance of vehicles. Intelligent vehicles must have the ability to adapt to these complex conditions. The last one is the requirement for practical use. Customers' need must be taken into consideration. To apply the navigation system based on computer vision to practice, the problem of the size and the cost of computer and CCD (Charge-coupled Device) must be solved firstly. The size of computer is becoming smaller and smaller nowadays, while the computing power needs to be growing and the price needs to decrease as well. Similarly, the price of CCD and image stuck is decreasing, whereas the speed of image acquisition needs to be faster and image processing capabilities needs to become stronger. With the rapid development of computer and electronic industry, in the terms of hardware, these requirements can be satisfied.
The functions of vision systems are environmental detection and identification. Compared with other sensors, the instruments of machine-vision can detect large amount of information and implement remote sensing as well. However, the computation needed to extract the objective and background from the complicated environment is so large that it would easily result in poor real-time system under current conditions. This problem can be addressed with some special image processing methods. For example, straight road boundary can be extracted from the image by using Hough transform. Then an appropriate prediction algorithm for path curvature should be adopted after straight road boundary being combined with the electronic map which is stored in the intelligent vehicle system. The speed of the identification of road markings can be greatly improved with this method, as well as the robustness of the vehicle system. Another method is to break down images of environment into various types. And different approaches of representation and methods of navigation would be used for different types, in order to avoid useless operations of information. As judging distance and speed of the obstacle through the information of single-frame image is very inaccurate, multiple cameras or sequence of consecutive images of a high-speed camera can be used actually. Consecutive frames of a camera also can be used to calculate the relative displacement of the target vehicle. To reduce measurement error caused by the instability of the environment, data from measurement should be processed by adaptive filtering.
In short, the combination of the computer image information and sensor technologies can quickly extract useful information from complex environments, thereby creating a reasonable planning and decision-making for drivers. Machine vision plays a significant role in detection of lane, vehicle following, obstacle, etc. And the sensor of machine vision is one of the most important sensors in intelligent vehicles research.
Only if accurate information is gathered by all the sensors and processed correctly will an intelligent vehicle system operate steadily. Therefore, studying how the information will be obtained, processed and analyzed through the sensors is very important. However, no single sensor can ensure the accuracy of the information at any time so far. By applying the technology of multi-sensor fusion, which information gained from a variety of sensors will be synthesized to make a comprehensive description of environmental characteristics so as to take full advantages of the redundancy and complementarity between multi-sensor data, accurate and needed information will be received.
Technologies for IAV
Recent applications of IAV
A framework of IAV
Intelligent vehicle decision and control technologies
Adaptive control techniques
An overview of adaptive control
Adaptive control has been developed for several decades. With the considerable advances of microprocessors, some parameter estimation techniques were presented for adaptive control schemes in the 1970s [book]. Since then adaptive control has been extensively studied and more and more research results have been reported. The formal definition of an adaptive controller can be found in Astrom and Wittenmark (1995):
"An adaptive controller is a controller with adjustable parameters and a mechanism for adjusting the parameters."
It can be inferred that an adaptive controller can adjust itself accordingly with the changes.
Generally, adaptive systems can be classified into two main groups: direct and indirect. Direct adaptive control, also regarded as model reference adaptive control, allows the controller parameters to be estimated directly . Figure 1 illustrates the basic scheme of model reference adaptive control.
It can be seen from the scheme that there is a reference model which gives the desired response to command signal. Comparing the desired response with the output of the system, the error signal can be obtained. The function of adjustment mechanism is to provide appropriate regulator parameters in order to reduce the error signal.
Indirect adaptive control is also referred to as self-tuning control when the parameter adaptation is asymptotically removed . In this approach, on-line identification of the parameters has been applied.
The objective of self-tuning control is to make the system adjust its parameters automatically. Figure 2 shows a general scheme of self-tuning control. The controller, process and the feedback constitute a classical control system. The outer loop consists of a parameter estimator and a controller design. As the process parameters are identified in real time, the controller parameters would be updated on-line.
Conventional feedback control system has the capability to modify its behavior as a response to the changes within the system and external disturbances to some extent. As the controller parameters are fixed, the performance of the system would be degraded significantly and even the system would become unstable, when the changes in the internal system or external disturbances vary dramatically. Therefore, it is appropriate for the systems which are required to maintain high performance in most of the time even if great changes happen to adopt adaptive control. Due to the rapid development of computer technology, adaptive control has been applied to practice nowadays. And compared with the direct approach, the indirect approach has been used more often. It is reported often that the applications based on single-input single-output adaptive control provide pronounced performance improvement for various systems.
Adaptive control has already been applied to dealing with the problems of intelligent vehicles. It is considered as a vital technique to improve the performance of intelligent vehicles. Development and specific applications of adaptive control techniques used in IAV are demonstrated in the following context.
Applications of adaptive control in IAV
There have been many researches revolving around the issue of how to apply adaptive control techniques to automotive industry in 1990s. The adaptive control theory makes a great contribution to the noise and vibration problems. The unique approach of integrating adaptive pseudo-feedforward and feedback control has enhanced performance of vehicles . In Holzmann et al. (1997) a possible solution, called Adaptive Cruise Control system (ACC), to the problem of designing special control systems to support drivers are proposed. Unlike conventional cruise control systems, whose only purpose is to keep a fixed speed, ACC slides from cruise control to a distance-control mode if a slower vehicle is detected ahead . In this text, a recursive parameter estimation method, known as one method of adaptive control, is used to deal with the changes. All changes in vehicle parameters can be detected on-line. Ackermann has separated the steering task of the driver into two major parts, a primary task called "path following" and a secondary task called "disturbance attenuation" . As the task of path following is much easier than the other one, efforts are mainly put into the task of disturbance attenuation. This task is assigned to an automatic control system since human being has to react for some seconds after the disturbance comes to him or her. Drivers do not have to pay much attention on disturbance attenuation while using this automatic controller, which can react to the disturbance quickly. From this point of view, adaptive control techniques that require response time for determining the controller parameters do not fit for dealing with the task of disturbance attenuation. Load adaptive real-time algorithms have been applied to the problem of a connected convoy of two vehicles. The whole convoy model takes account of ten degrees of freedom, and 14 non-linear differential equations represent the dynamics . Adaptive control algorithms are quite suitable for this real-time situation.
In the following part, two applications of adaptive control in the field of intelligent autonomous vehicles will be illustrated specifically.
An adaptive control algorithm for route guidance is explained in the first example. The objective of route guidance is to plan an appropriate route. It is obvious that navigation is one of the most important problems of intelligent vehicles.
Many algorithms that are used to solve the navigation problem have been put forward during past years. However, these algorithms cannot easily address the problems arising from huge road-network databases, constrained computation time, and rapid real-time database changes . In addition, even if a complex road-network map can be stored in the vehicle system, it would take much time to find the most suitable route, as well as to response to the rapid changes. Owing to the constraints of memory size and computation time, the algorithm for route guidance must be required to limit the size of map which is installed in the vehicle system. Moreover, the conditions of road and traffic are changing constantly. The route guidance algorithm must have a strong adaptability to varying conditions.
To address the problems above, a real-time adaptive control technique for route guidance has been developed. A hierarchical map structure is used to avoid excessive consumption of memory capacity and to reduce time-consuming searching in this algorithm. As presented in previous discussion, the key capability of indirect adaptive control is to estimate the controller parameters in real-time. The algorithm needs to run on-line in order to adapt to changing traffic conditions. Hence, adaptive control techniques can be made use of to tackle the problems caused by dynamic traffic conditions.
On the basis of the above factors, the characteristics of the algorithm can be concluded that :
The road network is displayed as various categories of nodes and the route information is gathered from hierarchical layered maps.
The database used to store maps is separated from other databases.
It only allows searching one layered map at an assumed time.
Different algorithms are integrated on the same search tree.
Having considered all the characteristics, a route guidance algorithm based on adaptive control has been depicted in the following diagram:
The second one introduces model predictive control, which is a heated approach to the design of self-tuning controllers, for intelligent speed adaptation.
A speed limiter installed in the intelligent vehicle system is used for intelligent speed adaptation. The speed limiter can modify the maximum speed as a response to the change of speed limit restriction, as well as inform the driver or take autonomous action when the speed limit is exceeded. There are two types of speed limits, fixed and dynamic, for intelligent speed adaptation systems. For fixed speed limits, the information that reminds the driver of the speed limit is gathered from a static database. On the other hand, the changing road conditions, such as road construction, traffic jam, accidents or bad weather, are taken into consideration in the dynamic speed case.
Initially model predictive control (MPC) has been used in the process industry. Later then it has already been applied to other industries successfully. Figure 3 shows the basic schematic representation of MPC. It can be seen from the figure that an on-line optimization is used to provide optimal control actions and a prediction model is used to identify the appropriate parameters for the controllers. Then an optimal control action is treated as input to the system. There is also a feedback from the system, in order to reduce the effects of possible disturbances and mismatching errors.
Although MPC controllers requires large amount of computation compared with other controllers based on indirect adaptive control, there are still numerous merits that the system can benefit from. Some of the advantages are as follows [book]:
A variety of processes including unstable, non-minimum phase, long and variable time delay processes, can be tackled.
Feed-forward control is naturally introduced in order to prevent the measurable interference.
A set of parameters "tuning knobs" in MPC methods allows the closed-loop performance to be adjusted. By changing these parameters, the MPC algorithm can result in some other control techniques.
"Program control mode" can be used to obtain smoother transient response, while the future reference trajectories are known.
In most cases, various versions of these controllers can be easily acquired.
If there are limits in the control signal, MPC controllers are more robust compared with other classic adaptive controllers.
A brief introduction of the application of MPC in the field of speed control is demonstrated in this part. The basic principle here is from . Discrete-time models are used for MPC. This application is based on organizing the traffic in platoons since it can dramatically increase capacity . A platoon is a group of cars with a lead car and one or more follower cars that travel in the same lane . In this case, the positions and speeds of the platoon leaders and the lengths of the platoons are contained in the state of the system. In addition, the speed limits for the platoon leaders are one component of the control signal. In the following stage, the controller uses an optimization to identify the control actions which are to modify the behavior of the system over a time interval. Next, an appropriate traffic model should be chosen among various traffic models for MPC prediction model. The trade-off between accuracy and computational complexity is the key factor that determines the model. The optimization algorithm needs to run repeatedly for each step. Consequently, the models that require detailed microscopic traffic simulation are not very well suited for MPC, while simplified models can be applied. However, the model can be replaced if it does not fit for MPC since MPC is a modular approach. Lastly, the criterions used to judge the performance of the system can be the total time spent in a traffic network. There exist many constraints, such as minimum space between two platoons, and speed limits, which should be considered by the MPC controller. In order to avoid the excessive time-consuming of the computation, control actions are taken in a receding horizon form. Specifically, the first sample of the optimal control sequence is accepted by the system and then the prediction horizon is shifted one step forward.
An overview of Fuzzy control
Fuzzy control, also regarded as Fuzzy Logic Control, which is based on the theory of fuzzy sets, fuzzy linguistic variables and fuzzy logic, is a kind of microcomputer control technique. Fuzzy control is also a kind of non-linear control method. From this point of view, it belongs to the intelligent control techniques. The systematic theory of fuzzy control has been developed during past years. Meanwhile, this technique has been extensively applied to different industries.
In fact, the fuzzy control system can be considered as a real-time closed loop system that depends on the experience of the operators. The input variables of the system can be calculated by the output values obtained by sampling. Next, the real input values should be transformed to fuzzy values. And then the fuzzy input values are used to compute the controller parameters within the combination with fuzzy inference rules. At last, the fuzzy values of controller parameters obtained from last step need to be converted back to real values as control signals. Figure 4 depicts a basic fuzzy control system structure.
It can be seen from the figure that fuzzy inference system is the core of the fuzzy controller which consists of fuzzy inference rules, fuzzification and defuzzification. The language used in the fuzzy controller is called as fuzzy linguistic variable, which is a unique method to describe the system behavior. The input variables in a fuzzy control system are in general mapped by sets of membership functions that represents the degree of membership (DoM), known as "fuzzy sets" . Given linguistic variables, the fuzzy inference system then makes decisions for control actions based on a set of rules. Here is a typical fuzzy inference rule [book]:
If DISTANCE is FAR then SPEED is FAST
where: DISTANCE is the input linguistic variable
FAR is a membership function, i.e. the attribute, of DISTANCE
SPEED is the output linguistic variable
FAST is a membership function, i.e. the attribute of SPEED
The IF part is called the aggregation and the THEN part is called the composition. And the aggregation part resolves multiple preconditions of a rule into one degree of truth using fuzzy logic operators, while the composition part assigns an outcome for each rule fired [book].
Why Fuzzy control
Conventional control systems are usually implemented PID (proportional-integral-derivative) control techniques. As a consequence, complicated equations are used to describe the relationship between the input and the output. Conventional control systems are advancing at an accelerated speed during these decades. The applications of this technique are increasingly prevalent in the industry. Why fuzzy control has been proposed under the situation that PID and other traditional control systems are so sophisticated has become a debatable issue.
There exist more and more requirements for the system performance due to the rapid development of technologies. To start with, the demand for flexibility is increasing because of rapid and widespread changes in conditions. This leads to the fact that a strongly nonlinear behavior would be presented by the system. Moreover, there is always a strong demand for new methods. The systems become highly nonlinear since more inner loops and functional feedbacks need to be added after adopting new methods. Furthermore, an integrated information system is required, which will demand a dynamic control system. In many cases, there is not a mathematical model for the control process. Or there is much demand for computer processing power and memory through using the conventional method.
The fundamental reason for introducing fuzzy control is to imitate the human behavior. From this point of view, fuzzy control is well suited when the system to be controlled is only partly known, difficult to describe by a white box model, and few measurements are available, or the system is highly nonlinear . In addition, fuzzy control system can be easily upgraded by adding new rules or add new features. In many cases, fuzzy control can be used to improve existing traditional controller systems by adding an extra layer of intelligence to the current control method . The design of complex systems has been facilitated by fuzzy control technique on accounts of implementing heuristic control algorithms that are formulated by natural language rules rather than mathematical models. It can also be applied to incorporate complex decision making and the analysis of numerical data into non-control systems .
An application of Fuzzy control in IAV
One specific application of fuzzy control to the distance and tracking control problem will be explained in this part. The objective of distance control is to maintain the proper space between two consecutive vehicles which are driving in the same direction, while tracking control refers to the way drivers change lanes. Both distance control and tracking control demand for the coordination of vehicles around. There is a tendency for all these tasks to be done automatically by the intelligent vehicle system nowadays.
Drivers usually depend on their own judgments of the distance with other vehicles without the help of intelligent control systems. These judgments are not that accurate, sometimes even incorrect. It may easily lead to traffic accidents. In addition, in today's non-automated traffic system, the coordination is achieved by signals, such as turn signals and brake lights of the vehicles, and social convention, e.g., providing room to a driver in the adjacent lane who indicates an intention to change lanes, supported by a legal code . It would result in terrible traffic jam or even accidents if the vehicles are not coordinated well. An intelligent control system can not only assist drivers in detecting the traffic conditions but also inform drivers when the space between vehicles reaches the minimum distance, for the sake of safety of drivers.
The advantages of fuzzy control have been given in Section 5.2.2. In a word, the fuzzy control technique can satisfy the requirements of distance and tracking control system. It can be concluded from above paragraph that there are complicated decisions for the distance and tracking controllers to be made. What's more, the distance and tracking control system has to deal with the dynamic traffic infrastructure.
In the next part, a brief introduction of distance and tracking control system structure is given. And the main theory here is referred to .
The experimental environment for this distance and tracking control system is based on grouping the vehicles into platoons which have been introduced in Section 5.1.2. The task of the leader is to provide route guidance and notifications that instructs the followers when path changes. For the followers, they have to detect the distance to the former vehicle and the signals from the leader. There are two main variables to be controlled, distance to the leader and alignment in trajectories, which can be treated as the input variables. As the speed and direction of the vehicle can be controlled through controlling the left and right wheel velocities in the test-beds, the output variables can be assigned as the left and right wheel velocities.
Figure 5 illustrates the thorough structure of the distance and tracking control system. According to the diagram, distance and alignment sensors provide crisp data of input variables that have been processed by the pre-processing to the fuzzification part at first. The fuzzification converts the crisp data into linguistic variables. In the following stage, the output linguistic variables are given by applying the fuzzy inference rules which consists of a set of rules that describe all the system behavior. The output linguistic variables are defuzzified to real data through the defuzzification part then. At last, the two outputs from the fuzzy controller are treated as set-points for left and right wheel velocities. There are two closed loop PI control systems for controlling the two velocities respectively. This is a typical example of hybrid control system since conventional control and fuzzy control have been both applied to this system.