What follows is the report generated by my blog host:

Here’s an excerpt:

A New York City subway train holds 1,200 people. This blog was viewed about 7,100 times in 2011. If it were a NYC subway train, it would take about 6 trips to carry that many people.

Click here to see the complete report.

This Article is an attempt to make users aware of some limitations when considering implementing redundancy in their PC based HMI/OI systems. This article suggests how redundant HMI systems can be made more fault tolerant.

Redundant Server Example

Redundancy Defined:

At the core redundancy just means that there is a duplicate that can prevent failure of the entire system upon failure of a single component.   Since we are limiting our discussion to HMI’s (“Human Machine Interfaces” for the purpose of controlling machinery or process equipment like part of an oil refinery), what we are talking about is 2 computers that can provide the operator an alternative method to control his process.

Simple HMI Redundancy:

One way of providing redundancy is to just give the operator 2 or more HMI’s to work with.  If one goes down, then another can be used while the failed computer can be rebooted, repaired, or replaced as needed.   In this scenario each HMI computer reads data from the various PLC’s, stores and retrieves the various HMI screens, processes it’s own alarms, historical information and other special services.  Also management of the changes is on a computer by computer basis.

Client/Server Relationship to Redundancy:

Most HMI software today that is capable of redundancy is split up into multiple software pieces (or applications) that talk to each other.  There is a single client software application and usually several server software applications making up the overall HMI application.  These software applications can run on the same computer or separate computers depending on various factors like the size of the system and system management needs.

Client View Node Examples

The client applications are the actual “view nodes” that the operators use to monitor and control their part of the overall machine or process.  The client PC’s get all of their data from the server software which are usually on a separate computer.  Typically there would be multiple clients accessing the same set of data for a common server application.  The server software is what actually reads the data from one or more PLC’s in the system and delivers them to the various client PC’s requesting this information.  Other server software may deliver the various screens to the client PC’s, while still others may deliver services like event and alarming notification or historical data to the various clients.  This architecture allows the client computers to be greatly simplified, simplifies changes, centralizes the database of screens, and centralizes/minimizes communications with the PLC’s and controllers.

When we talk about redundancy, it usually means providing duplicate client and server software for all of the various software applications that make up the HMI application.  The client side redundancy is provided by having multiple client PC’s available.  Typically the focus for redundancy and fault tolerance is on the server software applications.  The redundant servers are generally implemented on separate physical computers as the computer itself and the operating system that is hosting the HMI server application(s) is considered the weakest link in a modern control system.

What to Look For In A Redundant HMI System:

When one is looking for a redundant HMI solution it usually means so much more than what is included in the basic definition provided above.  Users are looking for fault tolerance with such features as:

1. Automatic failover if the primary system fails.  This is not always automatic.  This is sometimes called “hot redundancy”.   Also the system should be able to fail over from the secondary computer or server to the primary server if the Secondary failed when it was “active”.
2. Alert  the operator to a primary or secondary system failure.  This allows correction to the failed system while the other one is providing functionality.  Without this functionality both systems could fail before anyone would notice.  That defeats much of the purpose of providing redundancy in the first place.
3. Accurate self diagnostics to determine actual functionality of primary and secondary systems.  I’ve now seen a system that reports no problems but fail when switched from primary to secondary or back again.  This can give a false sense of security.
4. Allow end users to switch between primary and secondary servers.  A popular product only allows the programmer or someone with special knowledge to switch servers.  It is not an operator friendly function.  Programmatic access to redundancy objects that can be displayed on the user screens should be provided in the HMI development application.
5. Allow automatic switchover to the “standby” system to verify standby system operation.   This should only be allowed if the operator can switch servers manually in case the test does not work.  The alarm and Event should be logged if a failure is automatically detected and switch back automatically as needed.

Resources/Examples:

Managing Redundancy (or turning redundancy into fault tolerance): from “A Conceptual Framework for System Fault Tolerance” 3/30/1995: http://hissa.nist.gov/chissa/SEI_Framework/framework_16.html

Example Video of how a redundant server should fail over from NEC:  http://www.youtube.com/watch?v=OfKUybEsuaQ&feature=related

_______________________________________________

_______________________________________________

Updated: Dec 29, 2011  4:20 pm CST  |  Published: Dec 31, 2012  |  Created: Dec 29, 2011

BOP after removal for examination. Click to Enlarge

The Gulf Oil Disaster could have been minimized had the BOP (Blow Out Preventer) succeeded in disconnecting the oil rig from the oil well and closing off the flow of oil.  The investigation and report by Det Norske Veritas (DNV), the Norwegian company hired by federal investigators to perform forensic analysis on the Deepwater Horizon’s recovered BOP, indicates that when the blowout occurred the forces were great enough to dislodge and bend the drill pipe inside the BOP sufficiently that the BSRs (Blind Shear Rams) were not capable of centering the drill pipe for a clean cut when the BOP was finally actuated by the rig crew some 7 minutes after the drilling rig exploded on the surface some 5000 feet above the BOP.

This is leading to speculation that the inherent design standards for BOPs are not adequate to protect against this violent of a blowout.

__________________________________________________

• http://www.usatoday.com/news/nation/environment/2011-03-23-gulf-blowout_N.htm
• http://www.huffingtonpost.com/2011/03/23/bp-oil-spill-blowout-prev_n_839649.html
• http://www.nola.com/news/gulf-oil-spill/index.ssf/2011/03/blowout_preventer_failure_in_g.html
• http://www.whistleblower.org/blog/31-2010/1032-report-confirms-deepwater-horizons-qblowout-preventerq-failure-prior-warnings-from-whistleblowers-silenced

One gets a much better picture by reading several perspectives on this same DNV report.

__________________________________________________

BOP on sea floor "leaking" oil while robot attempts manual operations.

_______________________________________________________

What is PROFIsafe:  From Wikipedia:

PROFIsafe (PROFIBUS safety or PROFINET safety) is the first open functional safety communication technology for distributed automation systems worldwide. Its specification for PROFIBUS DP and PROFIBUS PA was published first back in spring 1999. It incorporates the knowledge of more than 25 renowned safety companies. Extensions for the Ethernet based PROFINET IO followed in fall 2005. More than 100,000 automation systems with more than 1 Million PROFIsafe nodes are currently in use worldwide (spring 2011).

In the past, safety automation had to be “hard-wired” and based on “relay” technology due to existing international standards. This changed with the advent of a new standard – IEC 61508 – specifying how microcontrollers and software can be used in safety automation. This triggered the development of PROFIsafe, which was to integrate safety into the existing standard PROFIBUS fieldbus technologies. PROFIsafe is designed as a separate layer on top of the fieldbus application layer and reduces the error probability of the data transmission to the level required by or better than the relevant standards. PROFIsafe messages are using the existing standard fieldbus cables in coexistence with the standard messages (“Single Channel”). PROFIsafe does not benefit from any error detection mechanisms of underlying transmission channels and thus supports the securing of whole communication paths, even backplanes inside controllers or remote I/O. PROFIsafe coined the term “Black Channel” for this concept, which now is adopted by most of the other safety fieldbusses. PROFIsafe can be used in safety applications up to Safety Integrity Level 3 (SIL) according to IEC 61508, Performance Level “e” (PL) according to ISO 13849, or Category 4 according to EN 954-1.

PROFIsafe is using expanded fault (errors and failures) detection mechanisms such as

• Consecutive numbering
• Timeout monitoring
• Source/destination authentication
• Cyclic redundancy checking (CRC)

PROFIsafe is standardized in IEC 61784-3-3. It also is a Chinese standard (GB/Z 20830-2007).

PROFIsafe runs its own web portal on www.profisafe.net (see Web Links) with more details on the technology and hints for device developers, integrators and end users.

The PROFIsafe standard is maintained, updated and marketed by PROFIBUS International, a non-profit organisation administered from Karlsruhe in Germany. PROFIBUS International is also responsible for the development of PROFIBUS and PROFINET, an Ethernet based fieldnetwork.

_______________________________________________________

Here are some quick links:

For a good general overview:  http://www.sci-eng.mmu.ac.uk/ascent/literature/documents/C01_PROFIsafe_System_Description_4342_Jul07.pdf

From Profibus International (PI): http://www.profibus.com/index.php?id=61.  Be sure to checkout the related links in the left side bar.

Vendors supporting it include:

 Siemens: http://www.automation.siemens.com/mcms/industrial-controls/en/motor-starter-soft/protection/safety/profisafe/Pages/default.aspx

Phoenix Contact: http://www.phoenixcontact.com/global/technologies/40872_42838.htm

_______________________________________________________

My thanks to Chad Tverberg at Control Assemblies for making me aware of this.

_______________________________________________________

_______________________________________________________

_______________________________________________________

For another openSAFETY standard protocol that can coexist on other networks including ProfiNet or ProfiBus apparently see: http://hennulat.wordpress.com/2010/08/10/opensafety-first-uniform-safety-standard/

_______________________________________________________

Updated: June 2, 2011  8:55pm  |  Created May 31, 2011

For my comments follow this link.

I also found a several additional resources by doing a google search for “PLC programming“.  A featured site that offers online and on site PLC program training is here.

PLCs.Net offers this syllabus for their video training course.  It appears to cover most all of the basic PLC programming concepts.

The basics include such things as the history of the PLC, how the PLC works, the basic functions, PLC programming languages, the numbering systems, addressing/tagging, various kinds of logic used in the PLC and how to put them together to build functional circuits or functions, subroutines, and program execution sequencing, shift registers, discrete and analog control, data manipulations, doing math, handling fast I/O (input and output events), and a few things about wiring up the I/O the right way.

________________________________________________________

Eric Byres

Eric Byres • As a person who has studied both OPC and Stuxnet in some depth, OPC Classic is a potential security issue IF handled poorly.

Unfortunately many companies do just that, leaving large numbers of ports open in order to make OPC work out of the box. This is easily addressed with modern OPC security technologies – for example, see “Securing Your OPC Classic Control Systems” on the OPC Foundation White Paper Downloads Area (http://www.opcfoundation.org/DownloadFile.aspx?CM=3&RI=781&CU=10).

Stuxnet complicates the problem by making extensive use of the same underlying protocols as OPC, namely RPC, for its infection exploits and its P2P network. To make matters worse, the Siemens PCS 7 system also replies heavily on RPC to function. You can’t just take an IT firewall and say block all RPC and expect your plant to run.

I could go on for pages on the RPC problem, but more details are available in the white paper “How Stuxnet Spreads” (available at http://www.tofinosecurity.com/how-stuxnet-spreads).

(FD: I was a co-author in both papers)

________________________________________________________

Some good thoughts and considerations on the value of systems integrators, weather or not you agree with the importance they place on certifications.

Main points include:

• Many client companies have turned their controls engineers into project managers with not enough time to implement projects.
• system integrators can flex needed resources for particular projects or within a project throughout the project life cycle.
• system integrators have a variety of specialized skills.
• find a system integrators and leverage them over the long term to get maximum value.
• develop CSIA (Control Systems Integrator Association) certifications that demonstrate skills to new clients.
• system integrators should know how to manage risk and have tools/techniques for allowing reuse of code and techniques that save time.

Basic Electrical/Electronic Formulas

Here is a handy Electrical Formula Wheel I found recently on the Web.

These are the basic electrical/electronic formulas I use on a fairly regular basis when doing electrical automation designs. 

These formulas are all derived from 2 simple formulas and these are:

V = I x R 

P = V x I

where

• V = Voltage (in Volts)
• I = Current (in Amps)
• R = Resistance (in Ohms)
• P = Power (In Watts)

All of the formulas in the Formula wheel can be derived from the above 2 basic formulas using a little algebra.  In fact this is what I typically do. I derive whatever formula I need starting with the 2 basic formulas and derive the needed formula.  Probably most electrical engineers do this as well I would dare say.  Still it is nice to see these formulas layed out in this wheel format.  It shows the power of learning just 2 simple formulas.

____________________________________________________________

For those new to the basic calculations used in electrical design or those that want a quick reference they can use throughout their careers I recommend “Ugly’s Electrical References“.  In addition to providing these simple formulas it also is a great reference for doing 3 phase power calculations and converted between single phase and 3 phase power equivalents.  I find myself turning to this reference over and over.  For example, It seems I regularly turn to this reference when doing power calculations since as a controls engineer I need to refresh myself when doing 3 phase power calculations.  Ugly’s also contains motor and conduit sizing tables from the NEC (National Electric Code) book as well as other valuable information, all in a back pocket-sized reference.

Updated: March 5, 2011  |  Created: Feb 14, 2011

Introduction

I found myself reviewing the basic method I use to tune a PID loop.  I would like to post my method here roughly as it appeared in my response to a question posed in the LinkedIn “Automation Engineers” discussion group linked in my “PID Tuning Discussion” post.

This was in response to tuning a temperature loop, but the idea is the same for most types of basic PID type loops.  A classic (or standard) PID algorithm (shown below) is assumed, (not the parallel or independent algorithm in the graphic above.. It just looked nice).

The classic PID algorithm

__________________________________________________

PID Loop Tuning 101:

We will need one key tool to do the tuning. We must be able to monitor the process temperature when a step change to the setpoint is made. We can do this by hand, recording the time and SP (setpoint temp) and PV (measured Temperature). Since this can take hours on a typical temperature loop some kind of trending tool will be most helpful. The purpose of this is to monitor the response to the step changes to the setpoint you will be making.

Step 1:
For starters I tend to use a gain of 1 or 2  (P = 1 or 2) and an I and D of “0″.  What a gain of 1 or 2 means varies by controller, so I adjust the P value until I see a full output from the PID controller (CV = 100%) when the error (SP-PV) is at a fairly large variation.  In the case of a 0 - 200 DegC temperature loop, adjust the P value until a 50 DegC temperature difference (SP – PV) equals 100% output (CV =100). 

Something like this is the result of Step 2 tuning.

Looking at the response will tell you whether to add more gain or less.  If the temperature stabilizes without overshoot (overshoot is a temperature that is above the final stable temperature.  On a trend graph this is readily apparent.  (See graph at the right for an example of overshoot).

Step 2:
Increase the “P” value until you get some overshoot where the temperature settles down fairly quickly.  (Quickly means within 2 or 3 overshoot cycles.  This can still take hours however).  If the temperature never stops oscillating then there is too much gain.

Step 3:
Reduce the gain by 25 – 50% from the value set in step 2.  This will make for improved loop stability and robustness to small process changes over time.

The reduced P and final tuning trend should look like this.

Step 4:
Add Integral or “I” to your PID controller to get to setpoint.  (Up until now we have not worried about getting to setpoint, only about the correct response and getting as much gain on the system as possible for stability).
The amount of Integral gain depends on the response observed in your system.  For set it and forget it tuning, most of the time one can just put in an integral term that allows the output to change by 10 – 20% of its total range in the response time of the system. 
The response time of the system is normally when the PV gets to 67% of the final value.  However I tend to use the full response time, so more like getting to 95% – 98% of the final temperature.  For example if it takes 4 hours for the temperature to stabilize after making a setpoint change, then adjust the integral term so it adds 20% to the output in 4 hours.

Step 5:
Basic tuning is done.  About 90% of the time, you’ll have a stable loop that gets to setpoint usually within the response time of the system.

Step 6 and beyond:
If this loop gets to setpoint to slowly, more integral can be added but this requires additional time which if you have it great.  I would advise to tune these slow types of loops as early as possible and to have the trend tools necessary to monitor any new tuning and step change responses while you are doing other things.  In this way you’ll have more time for experimenting to “optimize” the tuning before you need to move on.

__________________________________________________

Notes:

Disclaimer:  There are other ways to tune, some have been mentioned in the LinkedIn discussions.  This “closed loop” method represents little more then the basics.  However it is a common method that is used by experienced PID loop tuners to get close quickly when no other tools are available.  For this author other tools are typically not available, nor are they needed in most applications.

For temperature loops many in the LinkedIn forums have suggested some “D” to stabilize the loop.  I consider this advanced stuff and suggest to not add any “D” unless you live in the plant and are available to keep an eye on things, especially when something changes in the process operations.  Derivative is also often suggested if there is dead time in the system.  Again the same cautions.  I prefer to detune the loop even further over adding “D” wherever possible.

Also, if some kind of trending tool where you can monitor the temperature is not available, it might be better to start by tuning a faster loop like a pressure or flow loop.  With these you can see the response as you watch…. a trending tool while still helpful, is usually not needed.

Sorry if anyone thinks that you can predict the needed tuning parameters without any process knowledge.  Tuning a PID loop isn’t based on just numbers, it’s based on knowing your process.  Fortunately knowing the process usually just means measuring the response to a step change in the setpoint or some other “upset”, and we typically just measure it on the actual system.
For actual parameters, no one will be able to do any better than the P, I, and D parameters you will find in a working controller.

For those that can experiment with a working system, one can try changing the setpoint and measuring the response  This will tell you whether you can try adding (or if you need to reduce) the gain.

One more note:  If you change the “P” gain after you have added an I value, the I value will produce a different response.  In the classic PID control, the gain is multiplied by each of the 3 terms (P, I, and D).  Fortunately this is what is usually desired as the Integral and Derivative terms scale with the Proportional term.

__________________________________________________

Alternative Methods of Tuning:

One LinkedIn commentator has disagreed with me on my approach, preferring primarily to tune using the “open loop” method first.  I was unfamiliar with this method and I thank Michael Taube for educating me on this.  I certainly intend to try this method on an upcoming temperature control job where I’ll need to tune about 100 loops.  In controlling drive and motion control loops such as dancer and tension loops in web lines, I don’t see how one could tune such loops except using closed loop tuning.  Open loop tuning would make everything go out of control before you could get any useful data.

__________________________________________________

Additional Links to P&ID Tuning:

1. “Model Based Tuning Methods“   Contains a useful “PI tuning map” that shows the effects of increasing or decreasing P or I from an “optimal” setting.
2. “Open Loop & Closed Loop Tuning Rules” by Apco Inc.  I especially liked the concise summary on Open Loop Tuning in this article.
3. “PID Loop Tuning Pocket Guide Version 2.2” by ControlSoftInc.com;  Describes the relationship between PID terms, names and describes common PID controller implementations, describes basic closed loop and open loop tuning methods.
4. “Closed Loop vs Open Loop Tuning” by Michel Ruel of Top Control Inc.
5. “PID Controller“  Wikipedia Article shows the standard/classical and parallel/independent forms of the equation and some interesting history.
6. “PIDTuningClassical“  Wikipedia Article with some nice graphics and examples on tuning the classical PID algorithm
7. “PID Algorithm and Tuning Methods” by John A. Shaw, Process Control Solutions; Rochester NY;  Looks like a nice tutorial.

__________________________________________________

H a p p y   T u n i n g !

__________________________________________________

Updated: January 16, 2011  5:29pm CST | Published: January 12, 2011

The Blog-Health-o-Meter™ reads This blog is on fire!.

Crunchy numbers

A Boeing 747-400 passenger jet can hold 416 passengers. This blog was viewed about 5,300 times in 2010. That’s about 13 full 747s.

In 2010, there were 54 new posts, growing the total archive of this blog to 59 posts. There were 110 pictures uploaded, taking up a total of 10mb. That’s about 2 pictures per week.

The busiest day of the year was February 3rd with 124 views. The most popular post that day was PLC Programming Approaches.

Where did they come from?

The top referring sites in 2010 were linkedin.com, haroldennulat.net, nateholt.wordpress.com, en.wordpress.com, and lmodules.com.

Some visitors came searching, mostly for arc flash, iphone scada, scada iphone, entity framework, and wincc.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

PLC Programming Approaches January 2010
1 comment

SCADA PLC monitoring on the iPhone January 2010
1 comment

PLC or PAC? VFD or VSD? February 2010
10 comments

Designing For Arc-Flash March 2010

About The Author December 2009

__________________________________________________________________

The above content was automatically generated by WordPress.  It’s kind of fun to get a summary of the last year of blogging as well as the amount of traffic this site has generated.  My thanks to all my readers.