IT / OT Convergence: Part 2 The use of information technology in operational systems
This four part blog will provide my point of view on what IT / OT convergence is, why it is important to engage with the topic and where we are heading.
Part One, the introduction, can be found here.
This instalment takes a closer look at the use of information system derived technology in operational systems.
What, were they ever separate?
These days operational systems use a host of technologies that were developed with data and information management in mind. But this was not always the case. In the past operational systems often used technology that was developed specifically for the purpose of monitoring and controlling things. These include networks, computing devices and operating systems that met the real time and reliability requirements necessary to operate things. In Supervisory, Control and Data Acquisition (SCADA) systems the controller and the Remote Terminal Units (RTUs) were usually supplied by the same vendor and the protocols between them were unique and highly proprietary. This allowed vendors to make optimal use of limited network and processing resources.
The development of much faster and more real time capable processor chips and computers, together with the introduction of standards such as DNP3, allowed OT systems to become more flexible. Components from different vendors could, in principle at least, be used together. This flexibility happened at the cost of optimising the use of resources which was mitigated by the decreasing cost of these resources.
The move to more flexible, general purpose monitoring and control systems accelerated when technology developed for data processing systems became good enough to fulfil (many) operational requirements. Today most control servers run on standard server hardware and often on commodity operating systems such as MS Windows. The use of Ethernet (and even wireless) IP based networks is common and today many RTUs and PLCs (Programmable Logic Controllers) run LINUX based operating systems.
The use of IT based hardware, software and standards allows OT systems to benefit from the large amount of R&D that is invested in IT and the economies of scale that are achieved in their manufacture. This has the following advantages:
- Large supply chain and more vendor choice
- The use of software defined solutions that run on general purpose hardware allows smaller innovative organisations to compete
- Better interoperability between OT systems and IT systems
- Opportunity to use consumer grade components for less critical applications
- Larger skills base e.g. software developers and testers
- Use of IT software development tools (from design through to testing)
- Shorter development lead times
In short: More functions, quicker and cheaper.
The advantages outlined above however come with a number of challenges:
- While there may be many IT resources available in the market, OT systems need resources that understand operations. Many of these will not have the required skills to design and implement systems using IT.
- The ease of introducing new functionality can lead to more "bells and whistles" than required but with less reliability and more vulnerabilities
- The ability to build complex software defined operational systems makes testing more complex.
- Operational systems are often not connected to other systems (or have very limited connectivity). Many IT components rely on frequent updates and patches to remain stable and secure.
- Lessons learnt in IT system management (as described in ITIL for example) are not always implemented in OT systems as they used to require much less maintenance.
- Not all practices that are used in data and information processing systems are applicable or desirable in operational systems.
- IT components are often not designed to meet the non functional requirements (NFRs) of OT systems (e.g. environmental, reliability, real time determinism)
- Use of IT makes it difficult to maintain separation. For example operators may run office software on a Windows SCADA server - even though this is prohibited, consumer devices can be connected to networks (over Ethernet or WiFi).
- Traditionally OT systems are designed with a much longer expected life than IT components. This means that IT derived components may have to be replaced more often.
The challenge that is cited most often however is...
OT systems have relied on the following two attributes: They used proprietary networks and protocols which meant that there were very few people who knew how to attack such systems. Furthermore they were self contained with their own sensors, actuators, RTUs, PLCs, networks and controllers with few connections to other systems.
The first attribute, also referred to as security by obscurity, is only effective as long as the target is not very attractive. Malicious hackers that randomly attack any weak sites won't come across these systems. As soon as these systems become desirable targets to attackers who want to attack a specific installation (or set of installations using a specific system type), obscurity provides very little protection. The ease with which information can be found and exchanged over the internet has not helped.
The second attribute, physical separation, does make attacks more difficult, especially when combined with proprietary systems. The use of commodity (some times consumer oriented) components (PCs, Windows OS etc.) in OT systems is however reducing the protection offered by keeping systems disconnected. For example: USB memory keys were used to distribute Stuxnet. Furthermore systems that are thought to be isolated often have connections to other networks. Examples are components that have wireless network cards that have not been disabled, modems that allow the manufacturer to execute remote maintenance and human error in configuring networks. There are also good business reasons for connecting OT systems to IT systems; I will explore this topic in the next instalment.
IT systems that are exposed to the internet are under constant attack. Vulnerabilities are discovered, published and fixed via software patches. For systems to remain secure these patches need to be applied as soon as possible - something that can be more difficult in isolated systems that don't have access to other corporate networks or the internet.
This article shows how interested potential attackers are in infrastructure targets.
9 April was Internet of Things (IoT) day. More and more consumer grade devices are being connected directly to the internet. Many of these things are sensors that are used purely to provide information. I think that it is highly likely that we will however start integrating these information flows into all kinds of systems, including OT systems. Just as we have become very reliant on satellite positioning and navigations systems, more so than the original designers had intended, we are likely to see scope creep. Once the information is available it will be very difficult to ensure that systems don't become dependant on these information feeds and are not compromised should these devices fail, provide poor data quality or be compromised by cyber or physical attacks.
The combination of the factors above has led to a situation where OT systems are not designed, configured and maintained to the same security levels as internet connected IT systems. Examples of this include the lack of passwords or use of default passwords, insufficient security testing, weak or no encryption and signing and out of date software that has many known and published security weaknesses.
To summarise: Sensing and control systems have become more connected, less obscure and more valuable targets. They use more and more hardware and software components developed for information systems (IT) while still relying on obscurity and isolation. This poses a significant threat.
The economic benefits of sharing technology between IT and OT mean that this trend will continue and accelerate. Reliance on obscurity and isolation is misplaced though both can provide some benefits and are valid design considerations. This introduces risks but these can be managed.
All systems (IT and OT) need to be designed to operate in a given environment, meet both functional and non-functional requirements (including reliability and security) by making well considered trade-offs between cost and risk. They also have to be designed to be managed, maintained and changed. This is difficult. We do however have significant experience and knowledge in system engineering (much of which originates from the OT world), software engineering (mainly driven by IT) and the management of systems (both IT and OT).
We can optimise the use of our techniques and improve their efficiency but ultimately there are no short cuts. If we want safe reliable systems to manage our Smarter Planet (instrumented, interconnected and intelligent) then we have to engineer these systems using proven techniques and methods that have been developed in both the OT and IT environments.
PS: Thanks go to Andy Stanford-Clark (@andysc) for his review of and thoughtful input to this blog!