Historical reporting of security

Published: Last Edited:

This essay has been submitted by a student. This is not an example of the work written by our professional essay writers.

Historical reporting of security



Security information and event management (SIEM) technology provides real-time reporting and historical reporting of security events. SIEM products help to gather, store correlate and analyze data across a number of systems including applications, switches, routers, databases, operating systems, IDS/IPS and web proxies.

In the mid 1990's administrators were creating scripts to search through large quantities of logging data for certain events. An example given by Tipton and Krause (2008) is looking for multiple unsuccessful login attempts that are above a certain threshold. This approach however had a variety of problems:

  • The windows environment didn't offer the same flexibility as UNIX. The scripts were difficult to develop and couldn't be applied to many events.
  • Gathering information on events in this way is limited by the strength of the script. It is very easy for a developer to overlook or miss events.
  • The outputs of the scripts were placed in a file to be reviewed by a security administrator which often didn't take place until the next day.

The development SIEM products provided two major benefits to businesses; operational efficiency and management compliance. Manually managing the logs produced by firewalls and intrusion detection systems is a time consuming process and it is often very difficult to pick out relevant information with many events being missed and response times to events very high. SIEM products are able to manage logs and sort through them retrieving the relevant information whilst helping to solve the problem of storage. Using SIEM products to manage log files helps reduce the number of false positives security engineers have to investigate. This allows security engineers to concentrate on actual security events. The storage benefits of SIEM allow for historical data to be reviewed easily helping to meet the regulatory compliance imposed by governments.

There are four main functions of SIEM:

  • Log Consolidation/Aggregation - The normalisation of logs from different systems is a key feature of SIEM products. The variety of systems that produce system logs and different vendors make it impossible just to bring the data together. Once the data has been normalized it can then be fed into a correlation engine.
  • Threat/Event Correlation - The artificial intelligence that sorts through all the information coming from the different sources on the network. Event correlation provides the following features. It helps to reduce traffic by setting thresholds for certain alerts. By bringing alerts together from different sources it helps to make sense of seemingly unrelated events and tries to establish a relationship between them.
  • Incident management - the process that follows the successful identification of an attack such as notifying security teams via email pagers and phones whilst automated responses take place. Identifying attacks manually can take many days. In an environment when response times are critical the use of SIEM products can show administrators directly to the infected packet showing offers a large gain in terms of time and resources.
  • Reporting - SIEM products can produce a variety of reports from operational efficiency to compliance. Nearly all SIEM products allow users to create incident reports in a matter of minutes. Standard reports and custom reports offer a variety of flexibility for security managers.

It is the industry standard to deploy a variety of security products including firewalls, antivirus, Intrusion detection systems and VPN which can together produce a very large number of alerts which can be too much for a security team to handle manually. Also SIEM can monitor a company's security tools as a whole which allows for one central point of event signaling enabling additional defense to be deployed making the overall strategy stronger.

Analysis & Discussion

Our current demand for security for both internal and external threats is increasing both for compliance and security issues. The need for real-time analysis and response for security threats is a continuing growth. The two major devices used to control and report on these issues are Security Information Management (SIM) and Security Event Management (SEM) solutions. These devices are different in both response and information management and are described in more detail below.

Security Information Management (SIM) security solutions are to analyse and report from remote security systems and applications. These systems and applications are mainly other security solutions such as firewalls, IDS/IPS, anti-virus, VPN and other security systems. The main role of a SIM solution is to obtain information such as logs and alerts. Once the SIM has obtained this information, actions can be performed from the SIM console. Depending on the SIM solutions this data can be used for creating reports or even correlate the data to improve the quality of overall security information.

The correlation of data is found to be more beneficial as this does not only improve the quality of reports but also comply with a majority of important standards and increase the overall security of an organisation. These SIM solutions prime functions are to log all alerts and other information security products obtain and present it in a readable format. These reports must be examined by many governing bodies and also senior management. The reports are crucial to any system as this creates a readable document to all levels of management which are not technical. The data and reports held within these SIMs can therefore be used to monitor performance and security within applications and indentify which further added security will be beneficial to the organisation.

Security Event Management (SEM) security solutions primarily functions are not to correlate and report on data but to act to real-time security incidents. These solutions process real-time data from security devices such as firewalls, IDS/IPS, anti-virus, VPN and other security systems. Once this data has been obtained from other security products the SEM will then respond to the security incident and attempt to prevent the attack. SEMs are also used to perform other activities such as IT security, internal audits and other compliance regulations. SEM products are mainly a further development of protecting the network and system than how an IDS approach would work. The SEM product will survey the network and traffic to create a baseline of functionality and therefore react to changes of this pattern. These systems vary from SIM solutions as these respond to attacks rather than log them for later use. As an organisation with many employees this solution tends to be more beneficial as the threat level is larger of that of a smaller company. There are a majority of major players which provide solutions for the SEM market and have proven to be very effective in a working environment.

The need mainly for an SEM solution is the real-time analysis. A major problem within organisations is the time and effort put in by system administrators. It could take days before an administrators reviews and analysis all the reports and respond to these threats. With the SEM solution this aspect is eliminated, once an attack has been identified the system reacts by preventing this threat. For example, a SIM solution would immediately detect a brute force attack from an attacker but will do nothing to prevent this attack. Although an SIM will generate thousands of alerts it will not be detected by an administrator for days and by then an attacker could have access. An SEM however will process the alerts from this attack and block the attack in a matter of seconds. This greatly improves the security of a network and systems.

Both solutions react primarily by gathering data from security devices such as logs and alerts. All the major players within the security market make it almost impossible to feed events and alerts directly to these SIM/SEM systems. These platforms use different formats to present their data. This creates more problems when importing and analysing data on a SEM/SIM product. These products have to normalise all data product by these products without losing any key data throughout the process. Once the data is normalised for the SIM/SEM product it is fed into the Correlation Engine which is then processed and appropriate action is taken. If these security products were able to enforce a set standard for logging data and events then the import of data would be much quicker and more reliable. This would then eliminate the possibility of losing sensitive information which is held within these logs and events. The data held within the SIM/SEM solutions is crucial to both security and management of the network. These solutions can be held in a central location and feed responses from other external locations. This data is important to the organisation and the security of this information is very important.

The correlation of the data supplied is a simple task. All devices within a network produce alerts and logs, the main devices within a network are mainly routers, switches, firewalls, web servers, web applications and security products. The SIM/SEM products gain this information and compares or analysis these alerts. This automated process prevents an administrator to manually analyse and correlate these results which would take days. Compared to an automated approach which takes less than a second to compare and analyse. This saves on both resource and cost to an organisation and also prevention of an attack which would be identified almost instantly.

Although to an organisation this solution can appear priceless to preventing an attack this product is only as useful if properly configured. A badly configured SIM/SEM can result in many false positive or even miss an attack. Even though this equipment is impossible to configure to work 100%, many measure can be enforced to ensure the correct settings have been placed. These products are able to produce agents which can detect any reporting or database issues. Before the chosen solution can be placed compatibility issues must be checked with all devices. The product will only work if all other security and monitoring applications are compatible. Another aspect which must be considered is the rule set for both the firewall and IDS/IPS. There are no checks which can be placed by the SIM/SEM for these to be configured correctly. Rule-sets are behaviour based modelling to catch anomalies. A majority of security appliances are automatically configured with a basic set of rule-sets. No rule-set will be the same for all organisations; these are unique to their requirements and must be configured as such. Configuring rule-sets for such devices can be extremely difficult, correct training should be required before configuring all devices. If the rule-sets are configured incorrectly certain events will not be alerted or logged.

There are certain factors that organisations need to evaluate before implementing an SIEM system into their network. Companies will need to consider what the benefits of deploying an SIEM and weigh them up against the drawbacks. There are a number of different benefits that an SEIM can bring to a company's security.

One of the most critical operations an SIEM can provide to a company is to analysis and monitor events and filters meaningful data to a centralized log. For instance a small network only producing 10 events per second can accumulate up to a considerable amount of data very quickly and without some sort of software it is impossible for a human to sieve through these events and collaborate the important logs. Once SIEM software is implemented correctly it can substantially reduce event logs from an IDS, Q1 labs claimed to have an event reduction ratio of 500,000:1. SIEM's can then display these logs in a user friendly interface which enables system administrators to assess the events in real-time. This will also save time and money as it improves the effectiveness of the security monitoring. Another key benefit to the network that an SIEM brings with the ability to correlate data is the simplicity of monitoring a number of different devices at a single point. Most modern SEIM's provide a user friendly interface, which helps to simplify complex processes such as setting rules or reporting specific information. This can save network engineers a lot of time and make monitoring and defending of the system a lot easier. An SIEM also has the ability to monitor performances of devices on a network and if a network device was to ever fail then SIEM provides a function to send a support tickets by email or any other messaging service to an administrator. This can help administrators detect and respond to critical systems going down.

The ability to provide automated responses to firewalls and IDS's is obviously another huge benefit an SEIM can provide for an organisation. The ability SIEM's have of aggregating results in real-time allows the SEIM to provide real-time responses to fresh threats or attacks. This means with an SIEM in place it takes a lot less man power to protect the network form new attacks which ultimately saves organisation money and provides more up-to-date protection. SIEM's more based towards SEM technology than SIM technology tend to be superior in responding quicker and more accurate to attacks. Products such as CISCO MARS and CA are heavily SEM based and should be used if a company is looking for this ability. With this ability SIEM's also help provide protection from attacks like brute forcing accounts as it will picks up on the unusually traffic.

With all the devices been able to be controlled from a single console an SIEM can provide timed synchronization. For instance if the organisation needed to update the time on all devices or distribute a new rule set it could be sent automatically from the SEIM to all the devices on the network.

SIEM's offer companies long term storage of data which can be used for defining trends, provide historical analysis and support regulatory compliance concerns. This provides certain benefits for a company as it can help network administrators resolve where an attack came from or prove to authorities when and how data was stolen with archived logged data. SEIM's also helps to provide integrity to the data as it gathers information from a number of different devices and displays it all in a detailed layout. There are a few drawbacks of amassing a large amount of data, the first is the cost of storing the data, and the second issue is that for big companies storing a vast amount of data is the time it can take to process information as some SIEM's have been proven to take a considerable amount of time to acquire searched for information. "This was tested and proven in article called clear choice test"

Although there are many advantages for an organisation to buy and implement SIEM onto their network there are also a few obstacles / drawbacks to overcome. The first drawback a business would have to consider before buying an SEIM is the process of install the SIEM correctly. For SIEM to work properly it needs to know about all the monitoring devices on the network for example firewalls and IDS's, this can be increasingly difficult and complex for companies with a large network. Although more recent SIEM products likeQ1 labs and high tower have started to use an auto-detect feature which can analyze the traffic sent to them and recognize the device without manual input except setting up the device to feedback to the SIEM. Another drawback of SEIM is that they require constant monitoring from administrators which can take up a lot time and because of this some administrators tend to reduce the security threat level to reduce the amount of alerts displayed which can result in leaving the company's network vulnerable. The final and probably the most important factor a company will have to consider is the cost of implementing an SEIM, with some basic starting prices costing around £15000 and obviously increasing the larger and more complex the network is, an organisation will have to way up the pros and cons on whether to deploy it.