IT and OT – Part 3: OT is IT made by engineers

To finish for the moment the topic of IT and OT, started in the previous posts, I want to set some last points.

So, I propose the new definition – not because the world was waiting for one more, but because it combines the points important for me: “OT is a no-compromise closed- or open-loop process automation HW/SW solution, developed according to the end-user requirements”.

“No-compromise” means that one takes everything out of the existing technology, not being restricted by the “best-practises”, “guidelines” and opinion of the consultants. (And there are surprisingly few consultants in the OT-area…)

Closed-loop – in this context, it is a fully automated solution, like AGC (Automatic Generation Control – network frequency regulation).

Open-loop – means that qualified person (operator) decision is necessary, as in case of the classic SCADA system.

Process Automation – according to my opinion, it is always about process automation in OT. Data collection, working points setting, end-device control, but also data visualisation (remember the process visualisation out of 60-s years of the last century – the officers, drawing the enemy planes at the transparent glass screens?).

HW/SW – OT was initially less about SW – it was too expensive and not so robust. But very soon requirements of the flexibility caused the broader use of the classical computers with the specially developed software programs. Nowadays the balance between the HW and SW in the OT is very business-area dependent, but almost always the end-to-end OT solution includes specific OT HW (controllers, data collection boards, sensors, etc.).

Solution – it is almost always a special customer solution, based upon the standardised HW components and SW frameworks, being engineered to provide one integrated customer-unique solution.

The “end-user requirements” means the requirements of the end-user, not of his CEO or Enterprise Architect.

Of course, CEO can also have his requirements – I even proposed to make a separate chapter in the system requirements called “CEO Requirements”, so that one can put some representational, etc. aspects there. Why not – CEO is finally paying for the system, so he is allowed to have his fun. But it is normally not about CEO, CIO, unit or department managers, it is much more about the people in dispatching and in the field that will have to operate the system. (As opposite, to, let’s say, ERP Solutions, that are normally being bought by the C-level guys. And now ask the end-users of these systems what to they think about them).

Consequently, these are some further thoughts to the topic:

First: OT is much more bottom-up grown world, than IT. It’s the need to solve the problem, not the beauty of the strategy, concept, architecture or technology, that is in the foreground.

Second: OT tries to overcome the shortcomings of IT – normally in most efficient and effective manner. For example, necessity to transfer data with low Bod rates (100-s of Bit/s instead of today’s 100-s MBit/s) brought very efficient industrial data transfer protocols, that became industrial standards, but were never adopted in modern IT, that seem at some time point to stop managing computer and network resources (everyone, who looked at COM/DCOM protocol will understand me).

Third: OT is normally better tested. Only the OT people know such terms as FAT and SAT (Factory Acceptance Test and Site Acceptance Test, correspondingly) – when the developed and engineered solution is being tested first at developer premises (“factory”) and than on-site by the customer. Most of other users know the answer from the IT developer „It works on my notebook“.

Fourth: OT has no fear before using hardware and firmware. Being created by engineers, it does not stop before using self-developed or self-programmed hardware components to achieve the goal. Most of the software developers, even if they program the OT-like solutions, normally try not to think how the signal shall come into the PC or what will happen after the command leaves it. Sometimes it helps to take the SW developer to the running power plant, put him before the turbine, and explain, what all will fly around, in case the wrong command sequence will be executed…

Fifth: Its not about beauty, its about efficiency. But efficient solutions are normally beautiful.

Sixth: It is also not about the standards and the best practices. IT people seem not to ask the question “why to change the running solution” any more. They are all the time changing the things that are running (or may be were never running really good…). We had a long discussions with IT-architects, who tried to motivate, and later to force us to change the well-running OPC-based communication in our system to the “strategic” open-source ESB… This strategy was changed three times in three years since than, and the OPC system is running for over 13 years now.

Seventh: Clear end-to-end use case. Such a lot of times I have seen the promising IT technology, but any attempt to use it for the productive solution turned to be a flop – does not matter, how you turned it, there was always something missing hence not allowing to make it useful – a classical example, that the developers did not really understand the real-life application case.

Eighth: Making an OT solution out of IT is not always possible, and mainly involves the deep re-design and re-implementation at the lowest level. I just think about the attempt of one really big industrial corporation to make the OT-operational system out of Windows NT in early 2000-s. Combined with the attempt to develop the whole OT package based upon the state-of-the-art IT-architectures it almost ruined them those days.

And, finally: OT is normally being developed by engineers. If not – it starts suffering of all the problems of the IT.

I’m sorry guys, but I have seen in my life more IT people building solutions they do not understand, than engineers, doing the same 🙂

Because engineers are the ones who want to understand.

And with these thoughts, the final definition for today sounds: “OT is a no-compromise closed- or open-loop process automation HW/SW solution, designed and developed by engineers according to the end-user requirements”.

also on LinkedIn

IT and OT – Part 2: Terra Incognita

As we discussed in the previous blog, the border between IT and OT gets more blurred these days, but there is still one. Some of the colleagues, giving the feedback, pointed, that the previous post did not provide the answer to the main question – what is my opinion to IT and OT?

Ok, the previous blog also shouldn’t, but in this, and may be in the following one I will try to bring more beef in the discussion.

Firstly, the term “OT” seems to come from consultants (I even suppose to know who pretends to be the inventor of it), and is mainly used by the IT people, not the OT.

Looks as if the IT people and their consultants were moving through the endless terrain of the IT possibilities, until they came to the big river, where the people lived that already solved most of the problems, the IT could offer to solve… And more than that, they were not ready to replace the solutions, running over years, through the newest IT gadgets and best practices with bling-bling and mirrors on them.

“What’s that??” – asked the… you know, the Chief – the Chief of IT Office.

“These are… these are aborigines… let’s call them ‘OT’” – answered his consultant-man.

“Now I know how are they called, but what are they?” – asked the CIO again.

“Well, you know, the OT… I have heard, they are responsible for operation of all kinds of important things, and they seem to do everything differently”.

“And what shall we do with them?” – asked the CIO.

“Nobody will ever unterstand the OT, so let them operate their stuff as long as we are inventing their world new”.

So, or similar, should this story be, as most of the OT descriptions seem to be done by the people, that met something unknown and tried to describe it somehow. The majority of OT people have never defined what „OT“ is, and even worse, in most of the cases they do not know that this term exists.

If you look at the definitions of OT in internet you will notice, that most of them are:

  • Descriptive. They do not define, they try do describe what are we talking about. There is nothing wrong with it, but I’m normally rather sceptical about those descriptive definitions. They remind me the Plato’s definition of a Man as “a biped, without feathers.” And normally they reflect the poor understanding of the topic.
  • Archaic. This is one of the descriptions: “OT was used primarily in industrial control systems for manufacturing, transportation and utilities – and unlike information technology, the technology that controlled operations in those industries was not networked. Many of the tools for monitoring or making adjustments to physical devices were mechanical and those that did have digital controls used closed, proprietary protocols.” (source) Tears are coming in my eyes. In the days as this generation of OT came into operation (mid 70-s), the IT itself was busy with perforated cards and it was 20 years before invention of Internet.
  • Artificially reduced. “Operational technology (OT) is hardware and software that detects or causes a change through the direct monitoring and/or control of physical devices, processes and events in the enterprise.” (Gartner)
  • Bi-pole. “There is one pole, representing obviously simplified reduced IT and another pole, representing obviously simplified reduced OT”. And something in between that has to be converged via IoT. (like in the following example from the very interesting article on Industrial Ethernet Book)

Whats wrong with all that?

Ok, firstly, the free-form description of an object, business area or technology instead of definition brings us to the problem, that person, trying to work in this area never really understands what is it all about. There is a well-known example with SOA, where one has thousands of articles on the topic, but no real definition of it.

Secondly, the archaic presentation of OT ignores the last 5 to 15 years of development of technology in both IT and OT worlds (see also my previous post). It’s not now that the convergence between IT and OT begun. For the Utilities it begun around year 2000 on the technology level, and around year 2010 on the infrastructure convergence level. Most probably there are the business areas where it begun earlier and for sure there are areas that are facing this change only now – all depends upon complexity of the business processes and general needs of internal business integration. It also makes the OT to look primitive and complicates the further integration of both areas.

The artificially reduced definition of OT makes it easy to get the first understanding of the topic, but brings the huge uncertainty when one faces the real-life application environment of the complex company (of which the utilities, especially TSOs are the good example).

And, finally, the bi-pole description of IT-OT problem brings though the main artificially reduced aspects of the both worlds, but leaves absolutely blank the whole huge area between them.

And this is exactly the area where all the activities are being carried on.

This is the area where the so popular nowadays “IT-OT convergence” happens, this is the area where the IoT shall be applied, this is the area where the IT architects shall decide if the application belongs to the IT or OT world, and hence underlies this or that policies, restrictions, protections, etc.

To finish this post and to satisfy the wish of my colleagues to be more specific I would suggest the first draft of the first definition of OT:

“OT is what you need to operate your end-devices and everything you have to have to do it in efficient, safe, secure, and finally successful way”.

And a little bit descriptive, to add details:

“OT is a set of IT-based technologies and solutions used to control the physical devices – normally in the factory or in the field.

OT takes the IT technology as a basis and has no problem extending or changing it if this is necessary to solve the problem – IT is not a dogma for OT, but the direction to act.

OT solutions are highly robust, and were designed to run in the protected environment.

OT solutions include a lot of specific know-how about the business or process. (e.g. about power grid, chemical processes or hospital business)

Nowadays OT can include components, not directly involved into the monitoring and control of the physical devices, but necessary to support these tasks. (e.g. State Estimator and network security analysis for the electric SCADA).”

What is the difference to the classical definitions, like the ones above? It’s exactly that in my eyes the OT is since a long time not only primary control devices, RTUs/PLCs and SCADA, but also EMS/DMS, online analysis and expert systems, advanced visualisation and control – everything that is necessary to operate the system in, yes, „efficient and successful way“.

And in the next post I would like to address the topic of quality and fitting-for-purpose of IT and OT solutions.

also on LinkedIn

IT and OT – Part 1: know the difference?

One sees a lot of discussions about IT and OT, and there is a lot of information on this topic both in internet, and at the numerous digitalisation conferences, and on the consultants‘ slides.

At the same time, as my experience shows, often a clear understanding is missing talking about OT (let’s hope, that “IT” is clear to everyone 🙂 ) – at least, clear enough, to allow reasonable activities in the area, involving both IT and OT parts.

I spent about 20 last years of my professional activity on the border between these two worlds and with this post I wanted to address this question and may be get one or another opinion as a feedback.

In the former days, which lie for me around 15 years ago, the separation between IT and OT (ok, it was not called like that those days, but still…) was quite clear – everything you needed to do your real job came from the specialised companies, mainly the big ones, like ABB, Siemens, GE, etc. It ran under special operational systems and often on the special hardware. The people operating it had no clue how to start Windows NT, but new how to manage Unix or VMS. And it was what we will nowadays call „OT“. The real stuff. Period.

And IT, yes, it was also clear – it was Windows, and Office, and Outlook. Ant that was it – just the working place and a couple of servers to support the groupworking.

But in the following years a lot of things have changed. Fifteen years ago I’ve got a task to introduce the new system for the Swiss Transmission System Operator and it turned to be the first business-critical system on the MS Windows basis. The next ones followed out of the numerous projects of my colleagues. The delivery times got shorter, the budgets sunk dramatically, the user interface looked so that the normal person could use it, and the system stability… ok, they were mostly running.

So, after the fifteen years of development we have somehow the following picture, at least for the utilities:

  • most of the core business applications are running under Windows, or, even worse, are Java-based 🙂
  • most of the former big companies have less and less clue, what do you need, so your suppliers are widely diversified and do not fit under the former definition of the „industrial suppliers“
  • IT-Architects are struggling to find out which application is IT and which is OT
  • the failure of the non-critical business applications like Outlook or company’s Web-page can bring it to the crisis, not less and may be even more unpleasant than the down of some core OT systems
  • etc., etc.

and still, the IT/OT separation is somehow clear – or not?

What is your opinion?

And I will in the meantime summarise my further thoughts on this topic.