Comprehensible Credit Scoring With Ais English Language Essay

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Artificial Immune System is inspired from natural immune system. It is being used in many applications but in recent years, its use in data mining has been increased. In this paper, we describe a novel fuzzy AIS algorithm named CCS-FAIS applied to credit scoring problem. The algorithm represents its knowledge with fuzzy if-then rules. These rules have made CCS-FAIS human-understandable which are useful for experts. Our experiments were performed on Australian and German Credit Approval data sets. The proposed classifier has high accuracy and is competitive to several well-known classification systems.

Before credit scoring growth over the 25 years, banks used credit reports, personal histories, and judgment to make credit decisions. Credit scoring is an important task for lenders such as banks and financial institutes. They can evaluate potential risks for lending money to loan applicants with credit scores. The purpose of credit scoring model is to classify credit applicants to either a good credit group that is probable to pay back financial obligation or a bad credit group who has high risk of defaulting or becoming delinquent on the financial obligation [1]. It is one of the earliest financial risk management tools developed [2]. Its significance is more highlighted because of recent financial crisis.

Financial firms use various types of consumer credit risk management systems to reduce the misbehavior rate. Lenders must make decisions about granting credit to new applicants and dealing with existing customers. The first one is credit scoring and the second type of decision is behavioral scoring.

The benefits of credit scoring involve reducing the time needed in the loan approval process, saving cost average per loan, objectivity improvement which helps lenders ensure they are applying the same criteria to all borrowers [3] and easier supervising of existing accounts [4].

This work was supported by Iran Telecommunication Research Center.Development of credit scoring was started in the 1960s [5]. It has been widely studied in the areas of artificial intelligence, machine learning, and statistics. In the past, discriminant analysis models, logit models and linear probability models are used to develop these systems. Over the last decades, with growth of data mining, various classification algorithms are applied to this problem. One group of the novel methods that has been competitive to other classifiers are evolutionary algorithms. A more recent technique is Artificial Immune System (AIS). It is based upon models of natural immune system. In [21] AIS is defined as adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to problem solving. We use AIS and fuzzy logic (called fuzzy AIS) to present a credit scoring system.

The remainder of this paper is organized as follows: Section II explains the recent algorithms presented on credit scoring. Section III describes artificial immune system algorithm and the concepts we used in our algorithm. Section IV describes pattern classification with fuzzy logic. The proposed algorithm is presented in Section V and section VI reports the experimental results.

Previous Works

As mentioned earlier, in the past, traditional statistical methods have been developed for credit scoring. But the more recently developed data mining techniques such as neural networks, genetic programming and support vector machines (SVM) can perform better than traditional statistical methods. Choosing the optimal input feature subset and setting the best kernel parameters are two problems must have been solved to propose a SVM method [6]. Zhang et al. [4] and Huang et al. [6] used SVM for credit scoring. They show SVM has high accuracy for the problem.

Hybrid data mining approaches also have proposed for effective credit scoring. Yao [7] used neighborhood rough set and SVM as a hybrid classifier. In this classifier neighborhood rough set is used for feature selection. Zhang et al. [1] proposed hybrid model based on genetic programming (GP) and SVM. This model ran GP to extract if-then rules and it used discriminator based on SVM for remaining instances of dataset. Jiang [8] used decision tree and simulated annealing to build a model. In this paper, authors combine local search strategy of decision tree algorithms and global optimization of simulated annealing algorithm.

Exploring new techniques in credit scoring to improve the performance can save a lot of money. In recent years, many bio-inspired algorithms are presented for solving classification problems such as credit card fraud detection [9], credit scoring, security and other applications. Among these approaches, AIS is one of the newest methods has applied for the credit scoring purposes. Leung et al. [5] propose a simple AIS (SAIS) algorithm that adopts few key concepts of AIS (affinity measure, cloning and mutation). They found SAIS is a very competitive classifier.

Fuzzy logic has been applied to classification problems. Its advantages are powerful capabilities of managing uncertainty and vagueness [10]. Fuzzy classifiers generate a rule base structure. These rules are represented in linguistic forms that make them interpretable by users [11]. Experts can validate and correct the rules. This increases the interaction with users. Lei and Ren-hou [11] proposed a classifier based on immune principles and fuzzy rules. They apply their algorithm on 15 well-known UCI machine learning repository [12] data sets. They achieve high accuracy. The fuzzy AIS term in our proposed method (CCS-FAIS) is similar to [11]. The fitness function used in [11] is simple. The formula we proposed is described in section V.

Artificial Immune System

Biology has been a basis of inspiration for computational problems. Researchers have done much effort on the use of biological metaphors. Neural networks and genetic algorithms are the most famous examples. Artificial immune systems emerged in 90s as a new branch in computational intelligence [16]. Over the last decade, there has been growing attention in using natural immune system as a metaphor for computation. Learning, adaptability, and memory mechanisms are key features to make immune system a rich source for inspiration [13]. In [14] a general review of application areas to which AIS algorithm has been currently applied like pattern recognition, fault detection and computer security is presented.

Natural Immune System

The function of human immune system is maintaining the body immune and robust. The immune system is able to detect and eliminate some non-selves such as viruses and diseased cells. The mechanism of immune system is similar to 'hunt and destroy' which works on the cells of our bodies [13]. It has the significant talent of learning about the foreign substances called pathogens. The body responds to these foreign substances by producing antibodies, which can attack the antigens. Antigens are associated with the pathogens. There are two types of white blood in our bodies: B cells and T cells. A B cell holds antibodies on its shell which can identify the antigens invading the body. The matching between antigen and antibody is complementary and is similar to 'lock and key'. T cells do not interact to antigens directly. They circulate through the body and scan the surface of body cells for the presence of foreign antigens that have been combined with the cell. Then T cells bind to these cells until B cells helping to stimulate them [15].

Clonal Selection Principle

The basic features of an immune response to an antigenic stimulus are provided by clonal selection principle. The idea is that only those cells that recognize the antigen select to proliferate and others cannot clone. The properties of this principle are

A cell is duplicated and is subjected to mutation with high rates to form a new cell.

Newly differentiated cells transporting self-reactive receptors eliminate.

On contact of mature cells with antigens proliferation and differentiation occur [16].

One of the most famous algorithms proposed based on this principle is CLONALG [13]. Mutation is a key feature of this algorithm.

Pattern Classification with Fuzzy Logic

Let us assume that the pattern classification problem is a c-class problem in an n-dimensional continuous pattern space and the training data includes M real vectors from the c classes ().

Because the pattern space is, attribute values of each pattern are for and . In experiments of this paper (see section VI), we normalize all attribute values of the data set into the unit interval.

In our fuzzy classifier system, we use fuzzy if-then rules of the subsequent form for the n-dimensional pattern classification problem.

: If is and … and is, then Class with.

Where is the label of thefuzzy if-then rule, are antecedent fuzzy sets in the unit interval, is the resultant class (i.e., one of the given c classes), and is the certainty factor of the fuzzy if-then rule. It is a real number in the unit interval.

It should be noted that some antecedent conditions can be "don't care". Introducing "don't' care" conditions reduces the number of antecedent conditions of rules. These rules are more human-understandable than other rules.

We use a typical set of linguistic values in Fig. 1 as antecedent fuzzy sets. The membership function of each linguistic value in Fig. 1 is specified by homogeneously partitioning the domain of each attribute into symmetric triangular fuzzy sets. We use such a simple specification in experiments to demonstrate the high performance of our fuzzy classifier system, even if the membership function of each antecedent fuzzy set is not tailored. However, we can use any tailored membership functions in our fuzzy classifier system for a particular pattern classification problem.







Attribute Value





b) Attribute Value





Figure 1. The used antecedent fuzzy sets in this paper. a) 1: Small, 2: medium small, 3: medium, 4: medium large, 5: large. b) 0: don't care.

In our classifier system, the following steps are applied to calculate the grade of certainty of each fuzzy if-then rule [11, 17]:

Step 1: Calculate the compatibility of each training pattern with the fuzzy if-then rule by the following product operation:


Where is the membership function of attribute of pattern and M denotes the total number of patterns.

Step 2: Calculate the relative sum of the compatibility grades of the training patterns with the fuzzy if-then rulefor each class:


Where is the sum of the compatibility grades of the training patterns in class h with the fuzzy if-then rule and is the number of training patterns which their corresponding class is class h.

Step 3: Find class that is maximum.


If two or more classes take maximum value and training pattern compatible with the fuzzy if-then rule does not exist, the resultant class of rule cannot be determined.

Step 4: The grade of certainty is determined as follows:




Now, we can specify the certainty grade for any combination of antecedent fuzzy sets.

The task of our fuzzy classifier system is to generate combinations of antecedent fuzzy sets for generating a rule set S with high classification rate. When a rule set S is given, an input pattern is classified by a single winner rule in S, which is determined as follows:


That is, the winner rule has the maximum product of the compatibility and the certainty grade ().

Each fuzzy if-then rule is coded as a string. The following symbols are used for denoting the five linguistic values (Fig. 1):

0: don't care (DC), 1: small (S), 2: medium small (MS), 3: medium (M), 4: medium large (ML), 5: large (L).

Proposed Credit Scoring System

This section presents an overview of the proposed algorithm which is named CCS-FAIS. It is based on the clonal selection principle. The clonal selection principle is used to explain the fundamental features of an adaptive immune response to an antigenic stimulus. The main idea is that only those B-cells that identify the antigens are selected to propagate. The selected cells are exposed to an affinity maturation process, which develops their affinity to the selective antigens [11]. In this paper, no distinction is made between a B-cell and its receptor, known as an antibody, so that every element of our artificial immune system will be called B-cell.

CCS-FAIS algorithm uses a population of B-cells. Each B-cell lives in the population based on its assigned age. A B-cell dies whenever its age become zero. The objective is to obtain a set of rules with high accuracy. Each B-cell represents a rule. As we mentioned in section IV, each rule is coded as a string.

Fitness Function

CCS-FAIS have used the fitness function, which is computed according to (7) to (10). The basic formula which is named BasicFitness here is used in [17]. We add two terms to fitness in [17]. NCP is abbreviated of number of classified patterns and NMP is number of misclassified patterns. The final formula contains a term with rule length. The higher rule length the less fitness is achieved. It led the B-cells to short rule length. So, the final term is useful for comprehensibility. The less rule length the more understandable by human.


for each class do

Step 1-Initialization: Generate an initial population of B-cells from antigens in data set.

Step 2-Rule Generation: repeat

2.1-Clonal Selection: Select some B-cells based on their fitness to proliferate.

2.2-Hyper-mutation: Selected B-cells clone, actually cloning is hyper-mutation.

2.3-Replacement: Each B-cell matures and the ones which their ages reach to 0, die.

until maxIteration

Step 3-Rule Learning: select the best B-cell from the population of AIS algorithm. It adds to rule set if it improves the classification rate of rule set.

Step 4-Termination Test: check the termination conditions. If they are not satisfied, go to step 1 to find another rule.

Figure 2. An overview of CCS-FAIS classifier




CCS-FAIS classifier

An overview of CCS-FAIS classifier is presented in Fig. 2. The main loop of algorithm is that the classifier learns each class separately. This loop consists of 4 steps: initialization, rule generation, rule learning, and termination test. Rule generation phase uses AIS algorithm to find a rule based on the initial population. In rule learning stage, when a rule is added to rule set, the learning mechanism reduces the weight of those training instances that are covered by the new rule. So, in the run of next rule generation, AIS focuses on those instances that are currently uncovered or misclassified [17]. Weight of each instance at start of the algorithm is assigned to 1. Each step is described in details as follows:

1) Initialization: In this stage, a population of B-cells is generated. The number of initial population is constant. This number is a parameter of CCS-FAIS which is named initialPopulationSize. For generating a B-cell, an instance of current class from data set is randomly selected, then fuzzy terms for antecedent of generated rule is computed from each attribute value of the instance. The consequent part of rule is set to class of selected instance. Initial age of B-cell is another parameter denoted defaultAge. After generation of initial population, fitness is computed from (10) for each B-cell.

2) Rule Generation: In this step, a population of B-cells searches for optimized rule iteratively.

At first step of AIS, some B-cells are selected to be cloned. This selection is based on roulette-wheel selection algorithm. B-cells with higher fitness have more chance to be selected. The number of selection is constant. We define selectionSize parameter for the number.

Now it's time to proliferate for selected B-cells. Hyper-mutation is occurred during cloning process. A B-cell contains a rule and the rule has antecedents. A change to these antecedents cause change to B-cell which is mutation. Maximum number of simultaneous changes in antecedents of a B-cell is parameter which is named maxTermChangesNumber. We need to limit number of changes because modifying the most of antecedents of rule causes the change to the nature of rule. A B-cell is generated in a cycle, and then it clones and lives cycles in population. In its life cycle it explores a small part of huge search space. When it mutates, it alters the current search path locally but when the nature of the B-cell is changed, it startles and cannot use its past knowledge. The probability of changing an attribute value to don't care is a parameter that is named dontcareReplacementRate. Number of clones produced for each B-cell is another parameter which is called cloneNumber. The age of generated B-cells is calculated from (11). The formula controls the population size.


After cloning, parent B-cells (B-cells which is not produced in this iteration) are getting old. Some B-cells died because their age reaches to 0.

3) Rule Learning: When AIS algorithm is finished, from the final population, the best B-cell based on fitness is selected. The rule which is contained in B-cell is added to rule set. Then the classification rate of current rule set is compared to the old rule set which do not contain the new rule. Classification rate is calculated with (12). If the difference is higher than a threshold, the addition is accepted. The threshold is parameter with name accuracyThreshold.


4) Termination Test: If a stopping condition is satisfied, the learning of current class is finished, and the algorithm is going to learn the next class. If the condition is not satisfied, the algorithm tries to learn another rule with initializing a population for next run of AIS. We can use any stopping condition for terminating the loop. We limit the number of learned rules for each class. This is done by a parameter which is called maxRuleSetSize.

Experimental Results

We performed our experiments on two data sets, the Australian and German credit data sets, are available from UCI machine learning repository [12]. Australian credit approval data set contains 690 instances, 6 nominal and 8 numeric features, and 2 classes. All names and values have been changed to meaningless to protect confidentiality of the data. Class 1 is normal and has 307 instances and class 0 is abnormal and has 383 instances. German credit approval data set contains 1000 instances, 24 features which are all numeric, and 2 classes. The data set is more unbalanced than Australian, the normal class has 700 instances and the abnormal class has 300 instances.

We normalized features of both data sets between 0 and 1, and then we evaluate the accuracy of CCS-FAIS with 10-fold cross validation (CV) technique.

Table I shows the parameters specifications of CCS-FAIS for each data set in our experiments.

We compare our classifier with other classifiers presented in literature. Classification rate which is calculated with (12) is the comparison criteria to other algorithms.

Table II and III show the percentage classification rate of different classifiers when used on Australian and German data sets. The results show CCS-FAIS has high classification rate among other classifiers. The proposed algorithm is competitive to other algorithms. The other advantage of CCS-FAIS is comprehensibility of the algorithm. The extracted rules with CCS-FAIS make the learned knowledge of credit scoring human-understandable.

Parameter Specifications of CCS-FAIS


Value for Australian

Value for German








































Comparison of Correct Prediction Accuracy of CCS-FAIS with Other Classifiers for Australian Dataset


Classification Rate

Discriminant analysis [18]


Logistic regression [18]


Back propagation neural networks [18]


Hybrid neural discriminant model [18]


Quadsic [5]


CN2 [5]


ALLOC80 [5]


LVQ [5]




a. our proposed algorithm

Comparison of Correct Prediction Accuracy of CCS-FAIS with Other Classifiers For German Dataset


Classification Rate

Linear Discriminant analysis [7]


CART (feature selection with t-test) [7]


CART (feature selection with neighborhood rough set) [7]


RS+SVM [20]


RBF-SVM (feature selection with t-test) [7]




a. our proposed algorithm


In this paper, we propose a fuzzy classification system for credit scoring. Using fuzzy logic in AIS is presented in [11]. Credit scoring with AIS is described in [5, 19]. The proposed classifier is combined fuzzy logic and AIS concepts. A new fitness formula for fuzzy classification is presented. In this formula, we use three terms, NCP for increasing the accuracy, NMP for decreasing error rate and rule length for increasing the comprehensibility of the knowledge. One of the most compromises in evolutionary algorithms like AIS is balancing between exploration and exploitation. In this algorithm, exploring is done by mutation of B-cells and exploiting is applied with age formula. In (11) after cloning when fitness is increased, the age is increased. One of the features added to AIS in this algorithm is population control with age.

The future work of this study relies on improving the accuracy and performance of the algorithm. We can use other concepts in AIS like negative selection or immune memory.