Petroski Notes

Almost ready!

Random bits about the book "To Forgive Design" 2012 by Henry Petroski, and related ideas.

• Notes about the book

• Reliability and failure in electronics

• Abusing statistical distributions ... needs more work

• Vintage Tektronix oscilloscopes

• Robert Courland's "Concrete Planet"

• ideas for Cathodic protection of rebar

Book Notes

No complaints - just notes to help me remember what I found in the book, or what occured to me while I read it.

• P15 - Washington Roebling: "An engineer who has not been educated as a spy or detective is no match for a rascal." A mirror in computer security - protecting fragile software systems from malware. Similar to Bruce Schneier, many books about computer security.
• P17 - Chinese wallboard. Paul Midler's "Poorly Made in China: An Insider's Account of the Tactics Behind China's Production Game"
• P17 - Chinese PVC - polyvinyl chowmein. When replacing water main to front of house, be sure to specify American made. How? We are at the bottom of a hillside water district, the storage tank is 500 feet higher elevation, the feed pressure is very high.
• P27 - Challenger accident. Space shuttle problems stem from faulty system architecture, driven by unrealistic reusability goals. The heavy LOX tank must to be "above" the main engine thrust line. Insulation needed, but cannot be inside the tank or it might ignite. That puts large wings (needed for large crossrange landing requirements, specified by military but never needed) under the tank and its insulation. The solids were segmented to pass through rail tunnels from Utah - they could have been unitary construction if made east of the Rockies, but Jake Garn and Martin Thiokol helped politics triumph over safety and efficiency. The Atlas, with pressurized tanks and a vertical stack, is far more reliable and cheap to launch. Expendability? Atlas tanks were thinner than pop cans, the aluminum cost is tiny compared to the logistic costs, and an engine/turbopump capable of multiple launches must be heavier than a one-off, subtracting from payload fraction. The expensive reusable parts should be sensors and avionics, which should be plentiful for flight recording, and can survive reentry with the otherwise shredded first stage "balloon" tank. The key to low cost flight is not reusability of a few thousand pounds of metal, but logistics and voluminous automated data collection - !SpaceX.
• P37 - marble columns breaking in the middle, tied to spaceflight again. Euler's column formula limits the slenderness of rockets - otherwise they would be very tall and have very small frontal cross sections. Press reports of proposals for horizontally launched liquid fuel rocket systems show little understanding of stresses and ullage sloshing in horizontal tanks, and the need to climb above the dense lower atmosphere before pitching over and accelerating to orbital velocity.
• P38 - KC tension members breaking. Acoustic waves in cables reflect and double at the points of attachment. Unless the forces are perfectly steady state, the cables near attachments must be strongest, and gently tapered, with the risetime of stresses considered. The KC collapse occurred during a party, IIRC.
• P43 - iPhone . Steve Jobs only listened to his industrial design department. Style interfered with function on many products. The Newton, Siri software, and other botches were much more numerous than the iPod and the iPhone. Jobs was about to start a titanic legal battle with Google over Android when cancer took him. Only the young die good ...
• P72 - NY WTC truck bomb. Compare to James B. Stewart's "Heart of a Soldier", about Rick Rescorla, security chief for Merrill Lynch, who died saving thousands of lives on 9/11. The real hero of the day - had his earlier warnings after the truck bomb been heeded, bin Laden would have been killed years fore before 9/11. The truck bomb could have taken down the tower if the terrorists had placed it better.
• Natural disaster - the highest risk of natural disaster facing the US is the Cascadia subduction zone, currently estimated to have a 50% chance of a very large subduction zone quake in the next 50 years. Given the woeful seismic underdesign of Portland, Seattle, Vancouver BC, and cities in between (designed for magnitude 5, not 9+), large coastal resorts built on sandspits, and transportation systems easily severed by a big quake, it is possible that hundreds of thousands will die. The last big subduction quake in 1700 deposited sand on top of cliffs, and wiped out so many of the warlike coastal tribes that Lewis and Clark survived a winter without attack, a century later. The tsunami killed hundreds in Japan (which is how we know precisely when it happened).

• P77 - Nuclear plant. The Fukushima DaiIchi plant site was originally hilly. It was graded down to sea level for easier construction from barges. If the back up generators were up the hill instead of in the basement, and the electrical switching was above ground, the plants would have rode out the tsunami just fine. Units 1 and 2 were scheduled for cold shutdown the very next month. It would have been sooner, but safer replacement plants elsewhere in the TEPCO system were delayed by the permitting process.

• P93 - Simulation versus testing. Chip design suffers from this. We do a heck of a lot of simulation and modelling before building, but only nature can simulate everything. We are designing "all on one chip" systems combining radio receivers and processor clock drivers. Simulations based on simple models of interaction through the substrate (not unlike a waterbed with children bouncing on it) do not accurately model how the clock drivers can interfere with the receivers. So Intel wrote better simulations, and also designed the clocks so they do not emit noise in the same frequency bands as the receivers. Belt and suspenders. The Intel Sandy Bridge chipset simulations probably used tens of millions of dollars worth of computer time. The simulation did not detect that too much current draw occured when the USB ports in USB3 mode, leading to early failure. They only found this out after extended life testing, which was not properly analyzed until millions of chipsets had been built into motherboards. Learning from the FDIV fiasco, Intel spent almost a billion dollars replacing those motherboards (none of which had failed in the field so far).

• P94, picture - a Tektronix graphics display, probably a 4010 . There are 4 people lined up, it is not exactly clear who is JoMo doing the demonstrating, and who are the 3 named observers, but I assume he's the fellow on the right with the hand raised.

• Blind testing for cracks and fatigue failure: can this be done with transducers and acoustic propagation? A small gap is acoustically quite different than continuous metal, and I imagine the corner of a crack refracts high frequency sound waves in an obvious way, and that metal at the point of failure propagates sound differently than less strained metal. A few temporary clamps with transducers could shake (very small shakes) and observe, especially with some pre-modeling of the structural element suggesting where to look.
• P117 - plastic models. I sometimes built 20x larger models of integrated circuit structures, with plastic and copper tape and solder. Maxell's equations scale nicely, and I can measure everything with 20x slower (and much cheaper) equipment.
• P122 - picture. Could round aeroshells around girders reduce wind loading? Maybe. Or maybe it would just interfere with inspection. Perhaps the best inspection would involve hundreds of thousands of digital camera pictures, analyzed back in the office, much with automation, perhaps aided by cheap overseas engineers. The cameras, on long skinny poles, could reach inside shells, also controlling lighting of the photograph for better automated analysis.
• P133 - inner corrosion of wires in cables. Electrical test? An accurate measurement of end-to-end resistance versus temperature and ambient humidity probably could measure this. We can measure resistance with AC "4 point kelvin measurement" to 8 decimal places. I bet we could see individual wires break in real time (over minutes - to get 8 places you must do a LOT of signal averaging). Since current can find it's way though many paths, you might have to instrument all the paths to subract them from your measurements. But you probably want to know about their continuity, too.
• P139 - spacing trucks. Scales, timers, and a pullout lane approaching the bridge with signals directing trucks onto them if another truck is on the bridge. I imagine the problem is accentuated if wheel spacing matches structural spacing in the bridge deck. Extra resonances?
• Demolition - are there rich guys with more money than sense who would pay a million bucks to push the button that initiates the destruct sequence? Perhaps a lottery for the privilege. Perhaps additional filming rights for hollywood action movies. Whatever helps pay for a safe job.
• Failure Lottery - how about a betting pool, which bridge is going to fall next? The competition among civil engineers might identify some candidates in need of repair. If as a result of their diligent efforts, bridge is repaired before it fails, they get a compensatory prize, perhaps paid from additional funds added by the NTSB.
• We should have webcams on these structures.

• P168: deHavilland Comet - see Nevil Shute's "No Highway in the Sky", and the 1951 movie made from that with Jimmy Stewart and Marlene Dietrich . The accuracy of the failure prediction is silly, but Stewart as a materials engineer is priceless. Nevil Shute's autobiography "Slide Rule" is a must-read for anyone interested in the history of engineering projects.

• P173: Titanic. My grandfather embarked for the US on the Canadian Pacific "Empress of Ireland" in late March 1911 ( ticket here ). In 1914, the Empress collided with a collier in the St. Lawrence and sank, killing 1012 people, less than the Titanic's approximately 1600 casualties and the Lusitania's 1200, but mostly ignored by history. Not a structural failure, but certainly a traffic design problem.

• P180: Iron ring. Never saw one. But then, despite a Summa Cum Laude EE degree from UC Berkeley, in the state of Oregon I am not an "engineer" unless I take the PE test. Not relevant to my field (though I have other certifications).

• P199: Narrows Bridge. Gig Harbor is the upscale bedroom community for Tacoma, and is home to some very wealthy people. Friends live there. The bridges are pretty.
• P274: Good description of Gulf Spill. The energy industry employs fewer engineers per dollar than any other. The fewer they are, the more blame they get. Physicist Sir Michael Berry refers to similar asymptotic problem as "the worm in the apple" - if you bite into an apple and see a worm, you get upset. If you see HALF a worm, you get MORE upset, with your discomfort increasing the less worm you see remaining. But if you see no worm, you don't get upset at all. So if only one engineer was employed by the entire oil industry (with predictable bad results), the other millions of us would be truly evil monsters.
• Still, you would they would have tested the preventers better before deploying them. And had tools handy to cap the hole if they failed. We have armies ready to remediate diplomatic failures; why don't we have armies ready for other disasters? We have far more disasters (technological or otherwise) than we have wars.
• The biggest source of pollution in the Gulf is not oil spills - it is phosphate fertilizer runoff from the Mississippi, much resulting from corn grown for biofuel ( "Food'o'Fuel" - feed cars, not people)
• P312, picture of cranes. While I was reading this in a bus shelter, two 14 year old boys looked at the book over my shoulder and got really excited. One of them had already sold some artwork, and we talked about professional technical illustration, and I showed them some technical illustrations I did. Some kids still love engineering - there's hope!
• History of failure: In Japan, they say failure is a golden treasure. Back in the days of the first computer monitors, Japanese TV manufacturers made high resolution color CRTs on the same production lines as their less demanding TV CRTs. This complicated the lines, but the computer CRTs showed defects far more often than the TV tubes. It also improved adaptability to newer designs. Finding and solving problems on the difficult tubes made the ordinary tubes more reliable and yield better. Sometimes you do the hard stuff, at considerable cost, so the ordinary stuff can be cheaper and better.
• MORE LATER

Reliability and failure in chip design

Electronic product design accumulates failure experience more rapidly but more inconsistently than structural engineering. In a throwaway culture, a product is often discarded before it has a chance to fail. Exceptions include instrumentation (see Vintage Tektronix below), ships and planes, automotive, and communications central plant electronics, but even here the replacement cycles are rarely more than 20 years, and also obsolescence driven. Usually, the electronics is more reliable than the software running on it, so errors in software receive more attention ( properly so, though with less success ).

Electronics for implantable medical products (pacemakers, etc.) must have super high reliability, but medical volumes are small and the duration of use can be as short as hours (for catheterized imaging sensors, for example). The very high liability exposure often means that mainstream manufacturers actively avoid these markets - for example, Motorola sold piezo sensors for land mines, but refused to sell them for defibrillating pacemakers. Electronics for satellites require high reliability, but direct forensic analysis of failures is usually impossible; these failures are often inferred via telemetry.

Electronic systems are cheap and abundant enough to permit sampling and test to destruction - components and whole systems. Heat accelerates the chemical and thermal expansion processes that drive most failures, so electronics are tested with temperature cycling ( as much as -50C to 150C ) and month-duration high temperature testing ("life test") of hundreds of samples. Failure doubles for every increase of 10C, so a product baked at 150C for a month emulates a product operating at 50C for 80 years. These temperatures destroy or warp plastics, liquid crystal displays, etc., so forensics can be incomplete.

Sadly, some manufacturers of consumer electronics perform no testing at all, beyond making a few prototypes out of one batch of prototype parts and not finding egregious failures. We sometimes ironically call this cruelty-free - there's no testing. Like the animal non-testing equivalent, the customers become the guinea pigs. OTOH, companies like Apple perform hundreds of tests during product assembly in their Chinese manufacturing plants. Finding defects early, and correcting processes quickly, increases product quality and reduces scrap rate.

Some products are designed with built-in telemetry, making factory test easier, but very helpful for post-failure forensics. The IEEE 1149 family of test standards reduce the number of (unreliable) contact probes necessary to measure a product during manufacture. My company SiidTech designs and licenses identification circuits that are built into the chips, allowing failed chips to be compared to saved individual testing data on the wafer. Our clients use this to improve their tests (reducing future failures), detect otherwise anonymous chips whose parameters change during manufacturing (they will probably change past failure in the field), and even discard assembled chips that were neighbors to failed chips on the wafer (the neighbors tend to fail faster).

The greatest value of failure data is process improvement. Like all engineering disciplines, failed components teach us where manufacturing is inadequate. Non-failures teach us to be more aggressive, reducing costs and pushing the performance of future components. Cell phones and microprocessors are pushed to the limits - a cell phone can be more power efficient if smaller power amplifier transistors are used with higher voltages and temperatures. They degrade faster in the field, but most consumers lose, break, or replace their cell phones before this happens. Computer CPUs are stressed similarly; customers demand performance, and old computers grow uselessly obsolete, so computers are optimized for maximum performance over 5 year lifetimes. Hobbyist "overclockers" eke out a few percent more speed in while reducing lifetimes to months. The heat and increased current pushes metal atoms in conductors and creates cracks and voids - "electromigration".

Electronics have become so reliable that politicians push the other way. In order to prevent lead in landfills, RoHS ( Reduction of Hazardous Substances, "row-hoss") rules forbid lead in solder, moving to other "non-eutectic" tin alloys. These alloys do not flow like solder, and have high stress at grain boundaries in the metal. The material relieves stress by forming "whiskers" (long, spindly crystals) that reach out from solder joints, shorting to neighboring connections. These have caused failures in aircraft and satellites - crashes and derelict satellites don't add e-waste to landfills, either, perhaps an occasional pilot to a cemetery.

Perhaps the politicians are also increasing death tolls for structural engineering, with aesthetic and construction "convenience" demands. Airport noise reduction and fuel economy changes how aircraft engines and airframes are designed; the Rolls Royce engines for Boeing's 787 Dreamliner are engineering marvels, but they are pushing into new territory. International politics demand that components are made in China, Japan, and elsewhere, often by inexperienced manufacturers. The Dreamliner's carbon fiber wing roots turned out much weaker than planned, delaying deployment of the plane for years (fortunately, during a order-delaying recession). Boeing may pull all this off without increased failures in the field, they are very good engineers. But Airbus must push performance even more than Boeing to re-capture market. As these two giants vie for market leadership, we may learn we have pushed too hard as these planes age, fail, and passengers die.

Abusing statistical distributions

Most phenomena don't fit gaussian curves. Measuring a few samples, computing a mean and deviation, then extrapolating to large deviations drastically underestimates the probabilities at those extremes. Almost all real measured distributions from large numbers of samples have fat tails. On some of my circuits, I've built over one million samples (thousands of circuits per integrated circuit die) to characterize the extremes.

Nicholas Taleb's book "The Black Swan".

Bart Kosko's book "Noise".

More Later

Vintage Tektronix oscilloscopes

In the 50's through the 70's, Tektronix designed oscilloscopes to last a very long time. Some have, and are displayed at the Vintage Tek museum, between Beaverton and Portland. Normal museum hours are Friday and Saturday, but I imagine that special hours could be arranged for visiting dignitaries. During normal hours, many of the retired engineers who originally designed this equipment can be found in the back room, repairing it. A don't-miss opportunity for engineer historians.

The very durability of vintage Tektronix equipment means that there is a lot of it on the surplus market, and mouldering in instrument storage at hundreds of companies. One of my dreams is that universities could bring these old instruments and old engineers to campus for a two week "gross anatomy" course for seniors or grad students, taking these old instruments apart while explaining the engineering decisions that went into them.

I sometimes run a junior version of this, bringing a few hundred pounds of old electronics and a dozen sets of tools and safety goggles to a weekend academy setting, letting kids from 6 to 12 take stuff apart and see how it is put together (6 year olds love taking apart deskset telephones). I've had the dubious honor of watching a mom drag her daughter away kicking and screaming - the girl wanted to take stuff apart all night. Most kids don't care about how things are made. But the future engineers care a Whole Lot. Supplying them with scraps to take apart and rebuild (and teaching them to do it semi-safely) should be a priority for our profession. There is a lot of scrap wood that should be going the future structural engineers out there - how can we foster their projects while keeping the kids out of the emergency room, and ourselves out of court?

Robert Courland's "Concrete Planet"

I would love to see a review by a concrete engineer - this book seemed a little "selective" about defects in concrete. Book notes here

Cathodic protection of rebar

Concrete Planet makes much of the rusting of rebar in old concrete structures, and the trillions of dollars that may be necessary to replace thousands of them in the near future. I don't know how real the problem is. If it indeed a serious problem, we ought to be looking for ways to slow or stop this decay.

Rebar iron won't oxidize nearly as fast if it is biased cathodically. Too much voltage, and the electrode will emit hydrogen, which can embrittle the iron. Without knowing the voltage drop through the concrete, it is difficult to bias the iron properly.

A proposed invention

I'm not interested in patenting the following - consider it public domain. If it makes sense, please use it to protect America's aging reinforced concrete, saving lives and tax dollars. Perhaps a joint project between the CE and ME departments at Duke could get some papers out of it. I can imagine this turning into a little circuit board with a solar cell, a battery, and a small integrated circuit with a simple bluetooth radio transceiver for status logging and bias calibration, mass produced and added by the millions to reinforced concrete structures.

If you live in a place like Oregon with lots of rain and humidity, and have cheap telephone jacks with inadequate gold plating, you've listened to anodic oxidation of copper - it makes a hissing noise. These may be the little electrochemical events of copper ions oxidizing, small voltage spikes adding up to electrical noise.

The current drawn from a cathodic bias circuit on embedded rebar may make a similar noise, if the voltage is inadequate to prevent oxidation events. If the current is detected with a "virtual ground" amplifier, the capacitance of a few hundred feet of rebar can be nulled, and the audio band noise measured. Of course, the circuit will need a lot of filtering to reject radio noise and other antenna pickup.

I also presume that hydrogen generation is a higher energy phenomena, and has a different noise spectrum. So proper bias on a rebar electrode will be between the "copper oxidation hiss" and the "hydrogen hiss". It would be instructive to digitize the electronic spectrum of a piece of rebar in concrete as a function of voltage, and see if these hisses can indeed be distinguished, and a bias point chosen for the optimum tradeoff between hydrogen and oxidation.

If the voltage oscillates, we might learn about the migration of evolved hydrogen. Will it react with the iron oxide, reducing it back to iron and water? Can we learn about the depletion of alkalinity in the concrete around the rebar? We may learn to coat the rebar with materials that work in conjunction with cathodic protection to improve survival still further. These processes could be emulated at small scale in the materials lab, perhaps combined with high temperature lifetest acceleration.

I do not know how well this would work for large expanses of interconnected rebar - chances are there will be DC voltage gradients across the structure, so some portions of the electrode will be oxidizing while others will be reducing. While this technique might still help prolong the lifetime of existing reinforced concrete, it will probably work best on rebar electrodes carefully designed to match areas of similar ambient electrical potential and long term water infiltration and pH changes. So new designs will eventually get finite element electric field analysis, as well as mechanical analysis.

It may also be possible to find patches of damaged rebar by using large electrode plates on the surface of the structure to change the electric fields, and look at spectrum changes. Perhaps external patch electrodes could be added to parts of an existing structure's surface to remediate or stop damage. Ugly, but not as ugly as a bridge collapsed into a river.