The high-level component of IPSec

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Chapter 2 IPSEC


This chapter explains in detail the high-level component of IPSec and their working. The aim of this explanation is to make reader view the complete picture of IPSec Accelerator module and it's working. This module can install on Host-PC or Company's gateway router to provide security to VPN traffic. The security to IP packets depends on policies set in Security Policy Database (SPD) by system administrator. The cryptographic algorithm used for data confidentiality is AES (Advance Encryption Standard) and for data integrity is SHA1 (Secure Hash Algorithm). [1]

2.1 IPSec Framework

IPSec is a suite of protocols. Its architecture consists of three protocols as follows:

  1. Authentication Header[AH]
  2. Encapsulating Security Payload[ESP]

2.1.1 Authentication Header [AH]

Authentication Header provides data integrity and authentication of end user. It also helps in avoiding Replay attack by dropping former packets applying sliding window protocol and Sequence number.

AH header consists of following fields:

  • Next Header: It is 8-bit field which describe type of payload following immediately after Authentication Header.
  • Payload Length: it shows the length of AH packet
  • Security Parameter Index [SPI]: It is 32-bit field which is when compound IP address unambiguously describes Security Association with given data-packet.
  • Sequence Number: It is 32-bit field, whose value increment along with packets sent out. This is filed which is use to avoid Replay-attack.
  • Authentication Data: It contains Integrity Check Value (ICV) of the given packet. This field is 32-bit long, but value can be variable so padding is used.
  • Reserved: It is 16-bit field that been assign for future development. Normally it is set to 0.[2]

2.1. Encapsulating Security Payload [ESP]

ESP is one of protocol from IPSec suite. Unlike, AH this protocol provides confidentiality, authentication and integrity of packet. It can be used for only authentication or encryption or both. Figure 2.Encapsulating Security Payload header

ESP Header consists of:

Security Parameters Index [SPI], Sequence Number, Authentication field in ESP is similar to that one in Ah header.

  • Payload Data: It is the variable field, which contains the encrypted data that has to be transported.
  • Padding: It is used to pad the payload data to complete the length of block. It varies from 0-255 bytes.
  • Pad Length: It shows length of padding bytes used.
  • Next header: It describes the type of protocol of payload data of given packet. It is 8-bit long.

Both AH and ESP can be implemented in transport mode or tunnel mode. In tunnel mode entire IP packet is secured along with Inner IP packet and new IP header is attached to secure packet for packet routing. While in case of transport mode only upper layer protocols like TCP, UDP and data is protected, leaving IP header unsecured. Tunnel mode can be implemented in Host-PC or Company gateway, while transport mode can be implemented at host site only. It is mandatory to implement tunnel mode at company gateway routers. [3]

2.2 Working of IPSec

The architecture of IPSec processing consists of 4 important parts:

  • Security Association[SA]
  • Security policy Database[SPD]
  • Security Association Database[SAD]
  • Internet Key Exchange[IKE]

AH and ESP both take help of this functions to determine which kind of crypto algorithms to use for data security.

2.2.1 Security Association [SA]

Security Association fundamentally used to identify type of security to be implemented between to sender and receiver taking part in IPSec communication. It also has information regarding particular IPSec session. Same as AH and ESP, SA works in Transport and Tunnel mode. A single SA can be identified using three parameters as Security Parameter Index (SPI), Destination IP address and Security protocols. When there is communication in two-ways than 2 SA are used, as single SA can be used only in one direction. This SA also has the information of Key to be used in algorithms used for encrypting and hashing the given packet.

2.2.2 Security Policy Database [SPD]:

SPD is a crucial part of IPSec architecture. It is database in which all security policies are defined. It queried by outward processing and inward processing. Outward processing function queries SPD to determine whether particular packet needs to be secure or not. Inward processing query does security associate with given packet agree with policy defined.

2.2.3 Security Association Database:

SAD is also a database like SPD. It maintains the table of all active SA for outward and inward processing of packet. The entry in this database is feed in by manually administrator or automatic key exchange process like IKE. If the query for particular SA is not found in database than either packet is drop or two end communicate to create new SA. SAD is only queried if SPD decide that packet requires a security.

2.2.4 Internet Key Exchange [IKE]

IKE is a protocol used, within IPSec suite of protocol, to establish SA between two communicating entities when no SA is found in SAD. For implementation of IKE, Diffie-Hellman key exchange algorithm is used to establish secure communication. This process helps to negotiate between two parties on key to be used for encryption and hashing process, authenticate end users and determine which port to be used for further communication; this becomes phases 1 of IKE. In phase 2, both the users use the secure channel created in phase 1 to exchange the 2 Unidirectional SA created for given IPSec communication. This SA will be automatically feed in SAD database for any communication between the same end user here after. [4]

After packet goes from above function of IPSec, it needs to be passing through minimum 2 crypto processes. One would be used to encrypt the data to protect from eavesdropping and data confidentiality. Several algorithms like AES, DES, and 3DES are available for this process. And, second would be used to hashing the data for providing data integrity and authentication. Algorithms like MAC and SHA-1 are available for Hashing process. We selected AES and SHA-1 for our project and are explained in detail in next section.

2.3 Advanced Encryption Standard [AES]

AES algorithm is classified under symmetric block cipher, where sender and receiver use identical key for encryption and decryption process. It operates on 128-bit of data-length. It is also well known as Rijindael algorithm. They further classified as AES-128, AES-192 and AES-256, the size of key use for in encryption process in them are 128, 192 and 256-bit respectively. In AES basic unit used is byte, single entity comprised of 8 consecutive bits. AES perform its function on State, which is a 4x4 array of blocks, internally. [1]

AES is iterative algorithm, where single iteration is called round. The number of rounds to be performed depends on size of cipher key used for encryption process. Number of rounds to be performed for AES-128, AES-192 and AES-256 are 10,12and 14 respectively. All the rounds consist of four functions except first and last round in encryption and decryption process both.

For encryption these four functions are Add Round Key, Substitute Bytes, Mix Columns and shift row. First round only has Add Round Key process, while last round has all function except Mix columns. In decryption side for function are Add Round Key, Inverse Sub Bytes, Inverse Shift Row and Inverse Mix Columns. Similarly, first round only as Add Round Key and last round do not have Inverse Mix Columns.

2.3.1 Add Round Key

In this function subkey is add to state in round using bit-wise XOR. The subkey in each round is given by key expansion function using chipper key used for encryption process. The size of subkey is equal to state in all the rounds.[6]

2.3.2 Substitute Bytes/ Inverse Sub Bytes

In this function there is non-linear replacement of each byte by other byte using either S-Box (substitution-Box) or mathematical operation in finite field. This is two-sided process and is made up of two transformations as follows:

  1. Multiplicative inverse function over finite field GF[2^8]. Over here elements are mapped to itself.
  2. Irreducible polynomial used for this process is: m(x) = x^8+x^4+x^3+x=1

  3. Using invertible affine transformation determined by equation:

2.3.3 Shift Row/Inverse Shift Row

In this process the bytes in each row of the state are subjected to cyclic translation over numbers of bytes as shown in figure. There is no change in the first row. In next row single byte is to left-hand side. In third row 2 bytes are shifted to left side, similarly 3 bytes get shifted in last row. Inverse Shift Row performs same procedure as Shift Row, the only difference is that instead of cyclic left shift it perform cyclic right shift. [6]

2.3.4 Mix Column

This function operates on the state column by column, fusing four bytes in single column using invertible linear transformation. The output and input of this process are four bytes. Single column is counted as polynomial over GF[28] and is then multiplied modulo x4+1 with fixed polynomial given by a(x)={03 }x3+{01} x2+{01} x+{02}. In the Matrix form it can be seen as:

Inverse Mix Column operates in same manner as Mix Column function except fixed polynomial a-1(x) is used instead of a(x) and it is given by a-1(x)={0b }x3+{0d} x2+{09} x+ {0e}. In Matrix form it can be seen as:

2.3.5 Key Expansion

In AES, Key Expansion takes in cipher key and generate subkey used in each round. It generate 4(k+1) words where k is number of rounds. Key Expansion consist of SubWord () and RotWord (). SubWord () is function which take four byte input, applies S-Box to each bytes and produce four bytes output. RotWord () takes in a word [a0, a1, a2, a3] perform cyclic permutation and written a new word [a1, a2, a3, a0].

2.4 Secure Hash Algorithm-1[SHA-1]

It is a hashing algorithm, which has inputs of data as large as 264 bits and produces the result of 160 message known as message digest. It operates on 512-bit of data at a time. This message digest can be given as input authentication process like signature algorithm, by doing this we can speed up the functionality of process as working on small message digest is faster than working on whole message. On both the end same hashing function has to be used and used of these hashing algorithms provides data integrity because if there is data tampering than it will generate different Hash value. There is very little probability that different message producing same has value. The process consists of following steps:

  1. Append padding bits: Firstly, data is padded to make it equal to 448 modulo 512. Padding is generally to each packet. Hence padding value carries value from 1 to 512.
  2. Append Length: It is 64-bit block which contain the value of length of original data. It is also appended to message.
  3. Initialize registers: There are 5 32-bit (A, B, C, D, and E) register used to hold interprocess and final result of hash function. These registers are initialized to following values:
    • A = 67 45 23 01
    • B = EF CD AB 89
    • C = 98 BA DC FE
    • D = 10 32 54 76
    • E = C3 D2 E1 F0
  4. Process message in 512-bit blocks: this function of algorithm consist of four rounds, where each rounds is made up of 20 steps. Figure 10 explains whole procedure in detail. All the rounds have same steps except all uses dissimilar primitive logical function. Single round take inputs as 512-bit data block and initial value of ABCDE register and results are stored back in this register.
  5. The result of last round is added to the input to the first round to generate CV q+1.

  6. Output: SHA-1 produce the result of 160-bit after every 512-bit block been processed. Thus sha-1 operation in general is summarized as:


CV q+1=SUM32 (CVq, ABCDEq)


Where, IV=initial value of registers

ABCDEq= O/P of last round

L= # of blocks in data

SUM32 = addition modulo 232

MD= final result


  1. Provide protection versus brute-force attack by using long 4 bytes digest.
  2. It is more powerful against cryptanalysis
  3. Algorithm is uncomplicated and compressed compare to other algorithm.
  4. However, downfall of SHA-1 is it large numbers of round which make slower which can be overcome by hardware implementation.


  1. RFC 2401
  2. RFC2402
  3. RFC 2406
  4. wikipidia
  6. william stalling "Cryptography and Network Sceurity"
  • Figure 1,2 .
  • Figure 3,4,5,6:
  • Figure 7 : paper 2 alberto ferrante
  • Figure8.
  • Figure9-12:textbook William stalling