Managing L'unix

June 27th, 2009

The case of the missing Sun refugees

Posted by Paul Murphy @ 12:15 am

Categories: Disaster avoidance, Enterprise Policy, General, Sun

Tags: Job, Sun Microsystems Inc., Recruitment & Selection, Human Resources, Workforce Management, Paul Murphy

According to the Techserve alliance U.S IT jobs fell by another 34,800 jobs in May to end about 5% below the record levels recorded in November 2008. That’s about a fifth below the overall employment decline because the new taxes and regulations driving this are mainly aimed at the roots of the economy: primary production and manufacturing, where IT employment is both relatively limited and less variable than in secondary manufacturing and services.

We can expect, however, to see the IT percentage job loss number rise as the effects percolate through the economy and the next round of tax increases takes effect. There would have been, for example, only perhaps two or three IT staff among the 495 laid off in Tampa when Altadis was forced to close its cigar factory but many more will be affected as these people become net wealth consumers rather than producers.

It’s in this context of deteriorating employment opportunities for IT workers that I noticed something odd, or at least unexpected, about the little sample I get from the flood of resumes zipping around between placement firms.

The initial leaks about IBM buying Sun sparked an obvious and immediate surge in the number of Sun people casting around for new job opportunities - some senior people actually moved, many juniors quietly sent out updated resumes, and lots of people talked about poaching talent or leveraging contacts to create new businesses.

Many, presumably most, of these people are still looking and talking, but the surprise has been that the reality of the Oracle deal seems to be associated with a reduction, rather than an increase, in the number of Sun employees looking to jump. Given the state of the economy you’d expect more people to value the jobs they have, but while I’m told that attrition is up a bit at Sun, the number of current Sun employee resumes in recruiter in-baskets seems to be dropping relative to what it was in April.

This breaks the normal pattern in which a significant change in corporate direction or control triggers an early jobs panic followed by a gradually building rush to the exits as people who can find new jobs, do.

So why isn’t this dog barking? Obviously Sun HR values silence, and equally obviously people know new jobs are hard to get and so work to hold on to what they have - but I just don’t think that’s enough of an explanation, so how about this? Sun insiders know that the company makes the most efficient computers around, that rock is genuinely revolutionary, and that the Sun Ray looks better and better as Congress drives energy costs up and up - let me just quote Newt Gingrich on that, from yesterday’s Washington Examiner:

Estimates are that the Waxman-Markey bill will raise electricity prices by an astounding 90 percent. It will raise gasoline prices by 74 percent. It will raise the average American family’s energy bill by $1,500 each year.

And, far from creating jobs, experts predict that the global warming bill will destroy 1,105,000 jobs on average each year, with peak years seeing unemployment rise by over 2,479,000 jobs.

All in all, the bill is expected to reduce our gross domestic production (GDP) by $9.6 trillion.

That’s bad for almost everybody - but terrific for network computing in general and Sun in particular because the worse it gets, the better Sun will look and maybe, just maybe, enough Sun employees know that to explain the refugee flood that isn’t there.

June 20th, 2009

Seven post holes and one thought

Posted by Paul Murphy @ 12:15 am

Categories: General

Tags: Hole, Productivity, Processors, Semiconductors, Hardware, Components, Paul Murphy

As frequent readers know we got a malamute puppy earlier this year - and, at the time, it was about minus 30 and snowing so the fence I put up for his run was pretty basic. This week, that caught up with me: specifically we got a bargain price on a couple of wrought iron gates and the plan to improve that fence went from the distant future of the put-off-able to right now.

Doing it involves placing seven 2.5″ square steel posts, each of which needs about a 2′ post hole in ground packed down by the most recent set of glaciers, but since one of our neighbors owns one of those old manual post hole diggers - I thought “hey, how hard can this be?”

Very. So there I was sweating away, making about $0.50 per hour relative to renting a power post hole digger, and thinking about the old dictum that sixty men can not dig a single hole faster than one man.

Of course I had seven to do, and parallelism would have paid off big time -but even if I had any friends they’d have to be just about smart enough not to volunteer - so instead I took frequent breaks and used some of that time to read a bunch of recent computer science stuff looking specifically for new thinking on the problem of getting seven processors to dig one hole in significantly less time that it would take each of them to dig their own.

Mathematicians have worried about this kind of thing for hundreds of years in contexts like merging topologies; Morgenstern and Von Neumann briefly considered some related issues in their late 1920s work on game theory; and computer scientists of one form or another have nibbled at it since at the least the 1950s - all with essentially no positive results.

We’re really good at executing in parallel and reasonably good at breaking jobs featuring inherent parallelism into bits, getting them done, and then combining the results, but have made (at least as far as I can see) absolutely no progress applying multiple processors to unitary tasks -like digging a single post hole using one tool.

One thing I did to improve my rate of progress on digging those holes was to fill them with water whenever I took a break - and, assuming you stand back from the details far enough to see water as water instead of as some fixed quantity of H2O molecules, you can see just how analog computing would implement a solution to the problem.

Assuming you have a fixed flow rate and can only drop one hose at a time into each hole, you can set up to deliver enough water to fill one hole in one seventh the time it takes to fill with the hose you have simply by just slightly more than doubling your hose diameter.

Flow, in other words, is analogous to diameter -but controlling diameter is something we can simulate but not actually do in digital computing.

IBM’s cell is a Linux grid supercomputer put on a single chip, Sun’s CMT an SMP design for one chip, and both can, in terms of the water hose analogy, be seen as seven separate pipes delivering water between a shared intake manifold at one end and a shared delivery manifold at the other. The hardware and software comprising the manifolds for both of these things is getting better: thus both Solaris for CMT and IBM’s latest compilers for cell offer significant improvements over their predecessors.

Both, however, concentrate on making maximal use of inherent parallelism - which, for IBM means vector processing and for Sun means running hundreds of apparently concurrent jobs more or less independently - both offer order of magnitude throughput improvements over x86 (which you can think of as now offering multiple pipes, but no manifolds) but neither really gets one hole dug with one tool much quicker.

Both offer tremendous scope for future development of the more, smaller, faster varieties - for example, more memory, shorter connectors, and faster co-processors are all in the immediate future for both lines.

People are working on digitalizations of the simple analog approach: mimicking the effect of increasing the hose radius in software with some variations showing particular long term promise. My bet, however, is that it won’t be enough: that only revisiting the 1930s victory of digital computing over analog will ever allow us to build machines that we can grow to dig one hole, using one tool, arbitrarily fast.

June 13th, 2009

Murphy collezioni

Posted by Paul Murphy @ 12:15 am

Categories: Applications, Database Management, Enterprise Policy, General, Government, Linux

Tags: Airplane, Calgary, Corporate Governance, Linux, Networking, Business Operations, Corporate Law, Operating Systems, Software, Paul Murphy

Huh?

I spent last Saturday at the Wildrose Alliance (think Republican) annual meeting in Calgary. More than 300 people in one room almost all of whom agree with me on most issues. Wow! - I mean, seriously, how often do you think that happens?

Anyway, at one point (apparently apropos of an egg salad sandwich) a guy I’ve never seen before tells me he “digs Linux”. “Digs”? Say again? Anyway, I think there’s a bottom line lesson here: this world is getting way too small and too interconnected.

Interesting weather too: 30cm of snow in the Cypress hills, heavy snow and hail in Calgary -obviously global warming, as I think Ed Morrisey said somewhere, now really is something you can shovel.

The 21st Czar

This week was marked by a need to spend a little bit of time reviewing the mandates various national network control authorities have. In both Canada and most of Europe undisclosed police access to just about everything networkish is taken for granted, but the formal centralization of political power to hunt for and shutdown offending network operators or users is something few countries have undertaken - in fact, here’s a list of of the ones I found with internet security czars with that kind of mandate:

Communist China
Vietnam
Iran
North Korea
Cuba
Venezuela
The United States of America

Bringing down airplanes

Absent “black box” information - and one of the many things I don’t understand is why those are kept on board airplanes instead of having live telemetry recorded at an airline operated central site - the recent Air France crash now seems likely to go down to the unhappy coincidence of one too many natural factors.

On the other hand, I wonder whether airplanes will soon need much better internal electronic shielding to prevent the insane from bringing them down by frying the (non optical) pump controllers all fly by wire aircraft depend on.

Notice that I’m not suggesting this happened, only that the technology both in airplanes and at Radio Shack has evolved to make this possible.

Model based management

A couple of phrases from junk I was scanning earlier - things that had people “executing well against our model” in a context of “intensified mandates” and the desperate need to “implement the most agile BI strategies” - reminded me of a comment I’d seen, by “Jeremy”, on watts up with that:

The photo is priceless. This to me summarizes the complete lack of scientific approach to everything these days. Everyone sits round a PC and blindly believes whatever nonsense it spews out.

No first principles. No cause and effect. No understanding of physics. mathematics or even statistics. Just run any number of widely available computer modeling programs, fit the historical data and hey presto another science breakthrough.

It would be funny if it wasn’t so very pathetically sad.

(FYI: the article spoofed climate modeling by reproducing a press release from 2006 - proudly celebrating predictions that proved entirely wrong - but how many times have you seen otherwise smart people impressed with some “corporate cockpit” BI display where, in reality, the second letter should be an “S”?)

Fun with the drupal

I’m not sure I want to say that I, uh, “dig” Solaris and open source, but I did have an interesting demonstration of the power of the two come out of Saturday’s meetings. This particular agency has spent well into seven figures on a custom consumer information site that another agency now runs for it on a mainframe IFL. Response, of course, is pretty poor but the big issues she complained about are things that reflect the differences in thought processes and priorities between editors and data processing managers - and the thing that struck me during the conversation was that drupal will do most of what she wants out of the box - and the CCK (content construction kit) should make implementing the rest pretty easy.

So I did a very basic drupal demo (not, please understand, that I understand Drupal - in fact, it’s often so backwards to my thinking that it takes a very frustrating time to see the retroactively obvious) - downloaded and installed a clean 6.12, gave it a private database and apache virtual server address, added the modules she needed, added some time and date information others think she needs, copied and restructured some sample content from her site, and turned her loose - all in just under three hours and fully justified in terms of amusement value when she reported that her home access to the demo running on my ancient Sun 150 via Shaw cable was significantly faster than her access to the z9 IFL at the office.

Out of which an abberant thought: there are lots of editorial professionals out there facing daily frustration because the people who built their website tools simply don’t think like them (and, admittedly the notion that editors think is a bit overboard, but go with me on this, ok?) - and could therefore improve their own lives, along with system reliability and flexibility, simply by replacing all that eons evolved home grown stuff in favor of open source products like Drupal and Apache on Solaris or Linux.

And one more thing

From newsmax:

La. House Backs 15-Cent Charge on Internet Access

BATON ROUGE, La. — A 15-cent monthly surcharge should be levied on Internet access across Louisiana to fight online criminal activity, the House voted 81-9 Thursday, over the opposition of Gov. Bobby Jindal.

Rep. Mack “Bodi” White, R-Denham Springs, said he sponsored the bill for Attorney General Buddy Caldwell, to raise money to finance a division in Caldwell’s office that investigates Internet crimes, particularly online sex crimes against children.

The measure would raise $2.4 million a year for Caldwell’s department, according to a financial analysis.

“I don’t think that 15 cents per month is too much to ask for our children’s protection,” said Rep. Simone Champagne, D-Jeanerette.

So, quick question: if you just use your iphone -or cableco VoIP phone- for phone calls, do you pay the tax?

June 6th, 2009

Discussing Corruption

Posted by Paul Murphy @ 12:15 am

Categories: E-voting, Enterprise Policy, General, Government

Tags: Job, Information Technology, Recruitment & Selection, Human Resources, Workforce Management, Paul Murphy

As a consultant I’ve been asked for kickbacks a couple of times, but the overwhelming majority of the corruption I encounter among IT people is simple venality: agreeing with the guy you report to (or the people who control the local IT market) on something you know is against the interests of the organization paying both of you.

Even the simple stuff, however, quickly devolves into shades of grey when you look at actual cases. Thus the classic Nurenburg defense: following orders; seems perfectly adequate to cover contra-indicated actions taken as a regular part of your job, but the corollary about obeying those orders only when you don’t know they’re morally and legally wrong can kick in very quickly.

What, for example, should a fairly junior sysadmin, personally a Linux user, do when ordered to replace the OS and applications on twenty racks of Wintel servers with Red Hat equivalents if he knows that the order results from the boss’s amorous adventures with a Red Hat Consulting representative?

Look at a few real situations like this -and particularly the claims and counterclaims about nearly everybody else’s venality that go with a long term project collapsing into a blizzard of legal filings- and I guarantee you that you’ll soon be longing for the clarity you get with things like the traditional freezer full of cash or Holder’s use of his office to quash ACORN and voter intimidation prosecutions.

One example I’ve been thinking about recently is subtle, but carries a moral I thought worth sharing - it’s about a decent guy who simply didn’t know what he was getting into.

This guy (and like all my imaginary stories I’ve changed all the facts here except for the facts) has something over thirty years of “progressively more senior” IT experience that started as a COBOL programmer right out of college. In his penultimate job he was in charge of a fairly decent sized IT group babysitting a couple of z9s, the usual business applications clusters, and the thousands of pieces of wintel and PC networking junk that bigger organizations seem to accumulate.

But then he became a consultant, a senior “rent a manager” kind of guy who would (or so he was told) pop into a company or government group for a month or two at a time to straighten things out or keep them on track while they replaced some dearly departed. As he told me at the time, he wasn’t getting along that well with his wife, his kids have teenagers, and the freedom he saw in the lifestyle had appeal.

Four assignments and nearly a year later he found himself between rocks and hard places. On the positive side he’d found hotel living rather less fun than expected and was working at strengthening his bonds to home and family, but when he talked about the job every second word was about some moral dilemma or other.

What had really happened, of course, was that it had taken him three clients to wake up to the obvious: his employers didn’t see his job as providing services to clients, they saw his job as providing services to them through the sale of their products and services to those clients.

It was the third assignment that did it - his employers asked him to introduce and sell as the permanent IT manager an individual he believes had engaged in serious fraud a couple of jobs back and had since been traded around from one job to another by people whose primary concern has been to leverage loyalty to the same large hardware and services vendor he usually bought from for their own benefit.

According to him, he did the job; with success if no enthusiasm; got a couple of weeks off at the end, and then went to his fourth assignment with his eyes opened to the real nature of his employer’s expectations and a deep personal commitment to sending out resumes.

Since then he’s been benched for several months - with no new assignments, no job offers, and a pretty direct hint from the guy he reports to that employers who turn down his resume for interviews aren’t going to buy him as a high priced trouble shooter either.

Now he’s unhappy and unproductive, both unable to work in his field and unwilling to step out of it - but the moral of the story is, I think, quite clear: he should have terminated the relationship as soon as he understood it, because getting along by going along with moral corruption is a metastatic process: like a cancer, you either cut it out on detection, or it eventually kills you.

June 5th, 2009

JavaStation Reprise?

Posted by Paul Murphy @ 1:13 pm

Categories: Applications, Development, Sun

Tags:

A couple of weeks ago I contacted Craig Bender (”Thinguy”) to suggest that he and I work together to put some Sun Ray demonstration projects on the table for the developing Oracle display computing band wagon. He hasn’t responded - but I just got off the phone with a guy who’s pretty plugged in on the other side of this and don’t like the implications of what I heard.

The threat, I think, is that they’ll put a lot of weight behind the Windows java client for Oracle apps - basically resurrecting the ill fated 1990s java station idea without the MAJC hardware to pull it off.

The Java Station originally ran the 110(?)Mhz microsparc in classic thin client mode: download the app, run it locally, upload results. To say that it sucked is to be overly polite: a version of the WordPerfect suite customized for it was so slow it made Windows 95 look good, and the set up was extremely (ahem, cough) “network sensitive” - i.e. network congestion or slowing caused lots of hard to trace client failures.

I know no one at Sun listens, but here’s a heartfelt prayer from a frequent user: kill this idea before it gets up a head of steam and buries the best opportunity that’s come along since NCD for real network computing to succeed in the marketplace.

May 30th, 2009

Golf, Intimidation, and co-opting the boss

Posted by Paul Murphy @ 12:15 am

Categories: Applications, Development, Enterprise Policy, General

Tags: Wintel, Information Technology, Recommendation, Boss, Professional Development, Sales Strategy, Sun Solaris, Strategy, Career, Sales

One of the great frustrations of working in IT is that you very often know lots of things the bosses don’t, but they still get to review and possibly override any recommendations you make - even if those recommendations are directly in your area of expertise.

One result of this is that people who want to keep their jobs don’t repeatedly make recommendations they know the bosses are going to turn down - and that reality, in turn, affects how people trying to sell to your company behave.

Thus one of the things big, well established, sellers do is to play this out against both you and your boss -often meaning, in practice, that their mid level sales people go golfing with your bosses, and some juniors come round to your shop to listen attentively to your every need while quietly guiding you to issuing just the right purchase recommendations at just the right times to meet their boss’s quotas and expectations.

So, assuming you actually know better than to want the stuff you’re buying, how do you beat this strategy without prematurely ending your budding career in IT management?

One approach involves applying IT marketing’s 80:20 rule (bearing in mind that the numbers used in the 80:20 rule aren’t remotely point estimates - just magical stand-ins for comparing an assumed overwhelming majority to whatever’s left after that majority is accounted for - this rule states that 80% of new sales are made on the basis of the 20% of product claims that cannot be realized through your use of the product) by finding the intersection between the 20% that can’t be realized, and the companies your bosses want to buy from.

Imagine, for example, that you want to put a couple of your people to work with some new gear to develop and prove out some magical new application for the organization; think the right infrastructure for this is SPARC/Solaris and open source; but your bosses golf with some locally senior people from IBM and a guy who used to represent HP locally when that meant something positive, but now runs a consulting services company making its money by promising Wintel reliability and security.

Remember, when thinking about this, that there are two main reasons your bosses golf with these guys:

1. first, there’s social re-assurance: hanging out with these guys confirms to your bosses that they’re entitled to be your bosses and can understand the business of IT without having to deal with science, numbers, or facts; and,

2. second, they’re reacting to the same career threats you are - because their bosses; whether board members, appointees, or major shareholders; golf with more senior people at the bigger suppliers.

In this situation the dynamics are clear: the right technical proposal is a career limiting move for you, while one based on buying wintel gear from IBM and having the ex-HP guy front some people to work on them will cost the company much more to produce much less, but some of the instant and enthusiastic support this will garner from your bosses will rub off positively on you.

So what do you do? Try splitting the project into multiple pieces with the biggest chunk committed to the 80% you can achieve and the remainder divided in ways designed to gain senior executive support.

In the example, therefore, you would format your proposal to minimize the importance, value, risk, and cost of the 80% of the project you’re going to do internally on Solaris/SPARC, hype the skills and risks associated with the remaining 20%, list price it, and then carve it up between the Wintel wizards and IBM.

In doing this you’ll rely on part of the sales dynamic: the fact that the sellers will be perfectly aware of what’s going on and will adjust prices accordingly with the no hope seller cutting things to the bone, and the guys with the sure thing padding things out where ever possible.

As a result your 80:20 split can often be proposed as much closer to 20:80 in both costs and risks with your hacks limited to the 80% in boring low risk work they’re capable of and the 20% suppliers delivering the high risk, high cost, high excitement, stuff - and when you apply the same rates to cost out what the high price guys would want to do the 80%, your bosses can intuit huge cost savings from splitting the deal: meaning that approving the thing gets them what they want:

1. the social quid pro quo they deliver by hiring, or buying from, the guys who golf with them;

2. the frisson of well discharged executive responsibility they get from saving the company a few bucks on the boring, low risk, stuff; and,
3. the personal reassurance they get from having you agree that their outsider friends are smarter and more capable than you.

Obviously the hows and whats of this strategy are going to depend on the situation, but in general the bottom line on bosses who override technical recommendations on non technical grounds is that they’re usually reacting to either/or choices by preferring group and personal reassurance and emotional support over the possible corporate benefits associated with a decision they can see as technically risky - and that you can generally beat this by breaking each half of the either/or into the 80% that’s common between them but cheaper your way, and the 20% that’s sufficiently risky you’d rather have somebody else be responsible for it anyway.

May 23rd, 2009

Cooling future cost inflation

Posted by Paul Murphy @ 12:15 am

Categories: Applications, Deploying Sun Rays, Disaster avoidance, Enterprise Policy, General, Government, Sun

Tags: Processor, Data Center, Information Technology, Consolidation, Utilization, Server, Inflation, Data Centers, Storage, Hardware

Servers using Intel’s latest processors and the motherboards that go with them are among the most efficient, in terms of work done per unit of power consumed at full utilization, yet produced by the x86 industry.

Unfortunately a lot of that efficiency results from scale, and thus from workload consolidation and high system utilization - to quote Sun’s bmseer:

SPECpower_ssj results shows that servers (even those with the industry’s best power-management) running at low-utilization levels use many times more watts per unit-of-work than systems running at higher utilization levels. Datacenters can realize the biggest energy savings by running fewer servers at higher utilization levels (50% utilization or above).

Sun’s results on the 8GB (or 0.5GB/core) configuration show that running at 10% utilization requires 4.4 times more power per unit of work than running at 50% utilization.

4.4 times = (581 performance-to-power @ 50% utilization /133 performance-to-power @ 10% utilization)

Most SPECpower_ssj2008 are published on small-memory configurations that are much smaller than typical customer deployments. Sun is the only vendor to publish multiple results to clearly show effect of memory configuration.

A more normal-sized memory configuration of 32GB (or 2GB/core) uses 30% more watts than a tiny 8GB (or 0.5GB/core) configuration at 100% load. At active-idle the wattage difference is also 30%. Some competitors use additional configuration differences such as non-redundant fans, non-redundant power supplies, and single slow disk to further reduce the wattages and significantly improve SPECpower_ssj scores.

He’s complaining about competitors misleading customers, but the more important thing here is the extent to which utilization rates affect power efficiency - that 32GB, 16 core, Netra 4250 burns 225 watts doing nothing but making heat you also have to pay to remove -and that minimum only goes up by about 71 watts (31%) as you add workload until you max out the machine.

Now given the weathermen’s commitment to more than doubling American power costs over the next eighteen months, the obvious conclusions you should be reaching from these numbers include:

1. this may be the best argument yet for using OS ghosting in system consolidation and virtualization because, particularly for the Windows enterprise, this approach offers the lowest risk route to high utilization and thus lower power input requirements per unit of work done; and,

2. it’s also the best argument yet, particularly for the Windows enterprise, for replacing as many small servers as possible with virtualized instances running on much larger data center servers - because the limitations of Wintel software mean this is the only way to take advantage of the power savings available from larger, multi-core, gear.

Since these directions are consistent with what a large part of the industry wants to do for other reasons (mostly having to do with the consolidation of a different kind of power) the new energy taxes will, I expect, simply but radically accellerate an existing trend.

There are, however, several big negatives to doing this:

• First, services to users usually decline with consolidation - and usage flexibility, often the key to corporate operating success, usually flatlines just as soon as the data center gains enough control to become the tail wagging the corporate dog;

  Saving power dollars at the expense of business flexibility may, in other words, illustrate the adage about being penny wise and pound foolish.

• Second, data center consolidation within the client-server architecture does not address processor duplication - taking workload away from the desktop and giving it to servers doesn’t significantly reduce desktop operating costs. Consolidated client-server is therefore architecturally similar to the mainframe systems people hated in the 70s - just without the simplicity and reliability that came with IBM’s terminals.

  In other words, it’s not wrong to reduce server room power use, but the big opportunities for savings aren’t in the data center, they’re on the desktops - in fact if data center management measured system wide CPU utilization they’d quickly realize that every successful move to increase data center utilization through consolidatation actually reduces system wide processor utilization.

• Third, the long term IT staffing change is minimal in terms of head count but devastating in terms of skills availability and business continuity risk.

  Two things seem to happen:

  1. First, companies undertaking consolidation usually seek savings in help desk and related services but don’t realize them in the longer term as the absence of desktop change combines with increased communications complexity and user complaints to quickly drive those costs back up; and,

  2. Second, the increased IT focus on data center operations means that fewer people run more of the business critical systems, and, more importantly, that those survivors are increasingly cut off from the business.

  Thus what happens is that consolidation reduces the rate of externally driven systems change in organizational IT services without reducing the external presure for change. As a result business management works to find ways around real or perceived IT roadblocks and eventually, of course, IT management has to respond to the presure by doing something -but then discovers that there’s nobody left on the bench to do it, and, worse, that the people kept on for their fine work in the data center don’t speak user.

So is there a better idea? Something that responds effectively to exploding power costs while not incurring these negatives?

There is: and it’s the opposite of what most people will be doing: instead of consolidating everything to the data center, get rid of the data center. Move your processors and IT staff into user spaces, replace as many 80+ watt desktop computers with 4 watt desktop displays as you can, and use Unix, not ghosting, to run as many applications as you need on each box.

The downside to this approach is that it’s hard to carry out and unpopular with your peers, but the upside bottom line is that there are huge advantages for your employer in doing it - starting, not with the cost savings you’ll achieve, but with having IT actually work with and serve the people who make the money, not the people who spend it.

May 16th, 2009

Scorpions, Frogs, and Trench Warfare

Posted by Paul Murphy @ 12:15 am

Categories: Applications, Enterprise Policy, General

Tags: Consultant, Information Technology, Supply Chain, Channel Management, Strategy, Business Operations, Marketing, Management, Paul Murphy

Sometime before history started this distribution company bought a proprietary database machine with software originated by a comparable business out of Detroit and then added to and modified by others - including both a financial software house and the hardware vendor. The system worked, the company prospered, and nobody really cared about IT except at budget time when all agreed the system cost too much and did too little.

But that was then; a long way closer to now the company acquired a new executive vice president, IT consultants recommended by the auditors, a customer logistics committee, and who knows how many other players and agendas all of which eventually birthed an effort to clear out the boring old IT stuff by ringing in commodity computing, exploding productivity, and ever decreasing cost.

Great! Except, every time they had the new stuff working; it didn’t - and the solution always took time to find and money to implement: another FTE, more software licensing, more consulting, bigger racks, and new weaknesses in the company’s ability to hold and defend its data.

So eventually different consultants appeared muttering into their collective beard about clustering and scalability and packet filtering and “automatic intelligent data verification” and who knows what other drivel whose net effect was to reduce listener resistance while achieving pretty much exactly nothing in terms of getting anything to actually work.

Eventually the CFO ran for his career and the person left holding the bag resurrected the former IT head in a new role: consultant, now representing the company he used to buy from. After considerable cogitation (and 18 fully billable days), this worthy opined that maybe the whole thing was a mistake and they should just get a shiny new exemplar of that same proprietary database machine from his employer, have him assist first in hiring a new IT head and then in moving the old software from the now long obsolete box still running the business, and finally forget the whole thing ever happened.

And that’s just what they did -well, except for the “forget the whole thing” part. The formerly new executive vice president wasn’t about to wear the horns for spending almost six years, and nearly thirty million in actual checks written, to cause 40% annual turnovers in purchasing, warehousing, and financial management while angering every customer they had, every supplier they bought from, and every trucker they used - no, all this was someone else’s fault and by the time the company was done suing everybody in sight, it would be obvious to even the most benighted that he’d earned his promotion to the presidency through the pure brilliance of his efforts in keeping the company afloat through it all.

Now, this story is, of course, entirely fictitious -but on a personal basis I’d like to mention that even trying to imagine reviewing several inches of J’accuse, all in absurdly repetitive eight to ninty page clusters with cute little blue corners or other disfunctional fasteners, greatly decreases the will to live while giving rise to homicidal fantasies involving a lot -a lot- of screaming.

The company tried to hold the consultants and suppliers responsible, but I’m pretty sure the suppliers sold what they were asked to sell and that consultants hired for their flexible opinions did exactly what such consultants do - much like the scorpion in the story of the frog and the river.

So what really went wrong? I think the company put in place a person with a change agenda driven from a personal need to make bones - and that person not only decided to make that change by taking out a system that worked; but also decided in advance of any discussion and apparently mainly on the basis of drug store flyers and rumor, on the replacement technology.

Worse, senior management failed to stop it - none of them met their responsibilities in this context: not the former president, not the CFO, not the board: everybody knew the change wasn’t working, everyone could see costs mounting and middle managers bailing out, but not a single one of them stood up for the company, its customers, its employees, or its shareholders.

That, I think, was the critical failure here: because this was a story of escalating commitment by a naked emperor - a story with a simple lesson: if the first million (or, in Pelosi’s case trillion) leaves the kind of smelly hole you get from pulling down an in-use outhouse, everyone remotely involved has an absolute duty to scream in protest before the next millions get dumped down the sewer.

May 9th, 2009

NCIS: Episode #140: Breakout

Posted by Paul Murphy @ 12:15 am

Categories: Applications, Deploying Sun Rays, Enterprise Policy, General, Government, ~ Special Series ~

Tags: FAA, Phone, PC, Server, Hacker, NCIS, McGee, Abby, Gibbs, Ziva

As regular readers know my favorite TV series is NCIS - but I wasn’t terribly impressed with the recent two part “Legend” episode designed to test market a spinoff NCIS - OSP (Office of Special Projects), apparently to be set in Los Angeles.

What I liked least about it was the bare faced product placement for Microsoft’s table top computing - really just an iPhone like gesture interface for large screens. In the lead up to this, McGee describes the OSP as having “cool toys”, and in the actual program the whole gesture computing thing plays about the same role the Pentagon’s 300 page daily targeting orders did in Vietnam: not even usable as toilet paper.

Of course whining about the technological illiteracy and financial mores of the TV entertainment industry is worse than than pointless - so in my continuing quest for humility, I thought it would be fun to produce sketches for my own NCIS script - one intended to set things right….

But first, a little external context setting - The introduction from a May 7/09 report by CNET’s Elinor Mills

Report: Hackers broke into FAA air traffic control systems

Hackers have broken into the air traffic control mission-support systems of the U.S. Federal Aviation Administration several times in recent years, according to an Inspector General report sent to the FAA this week.

In February, hackers compromised an FAA public-facing computer and used it to gain access to personally identifiable information, such as Social Security numbers, on 48,000 current and former FAA employees, the report said.

Last year, hackers took control of FAA critical network servers and could have shut them down, which would have seriously disrupted the agency’s mission-support network, the report said. Hackers took over FAA computers in Alaska, becoming “insiders,” according to the report dated Monday.

Then, taking advantage of interconnected networks, hackers later stole an administrator’s password in Oklahoma, installed “malicious codes” with the stolen password and compromised the FAA domain controller in the Western Pacific Region, giving them the access to more than 40,000 FAA user IDs, passwords, and other data used to control a portion of the mission-support network, the report said.

They use Wintel - and their security failed, gee, ya know, go figure.

So:

NCIS: Episode 140: Breakout

SCENE 1

(NCIS Headquarters, bullpen; Tony, seated, is on the phone, McGee is standing with Abby in animated conversation, Gibbs enters from stairway above accompanied by a low thirties, marginally overweight, 5′6″ brunette).

Abby: Gibbs, Gibbs, have you heard from Ziva?

Gibbs: “No, Listen up, Ruth O’Brien is TDY DIA investigative services. With us.

Tony: (hanging up, turning, standing up) “Where’s Gibbs? - oh, sorry Boss, Dead woman in our parking lot. South doors.

Gibbs, “Grab your gear, lets go; Abby tell Ducky”

Scene 2

Dr. Mallard squatting near face down corpse of late 20s woman wearing conservative office clothes, lying on grass beside a walkway, in plain view from the sidewalk but hidden from the parking lot by a large tree and from ground floor windows by bushes.

Tony is making sketches, McGee taking photos, Ruth is walking away toward the parking area, Gibbs stands beside Ducky. Jimmy is standing looking down at the back of the dead woman’s head.

Ducky: “Let’s turn her over”

(Jimmy helps - she has an NCIS badge on her shirt.)

Ducky “Margaret Anders, a civilian, worked for us - NCIS internal audit.”

Gibbs: “What killed her, and when?”

Ducky: I’ll know more later, but I’d say this poor girl suffered a severe blow to the back of the head - this contusion here (pointing where part of her scalp has been ripped away) - and then the killer put something around her neck - see the wide red mark all round (pointing) and strangled her. Took terrific strength to use something that wide - he may have, (Ducky rolls the body back over, lifts shirt from the bottom, we see bruising) yes, I think she was unconscious from the blow and he stood on her back to pull the noose tight while she died.

As to when, I’d say not more than twenty minutes ago; perhaps she had a 9:30 appointment in the building.

Ruth: (entering from the walkway) “Gibbs, no security coverage, that tree (pointing) blocks the camera for just this bit of the walkway.

Tony: I’m sorry, who are you again?

Ruth: “Ruth, Mr. Dinozzo; my name is Ruth, I’m filling in for Ziva.

Gibbs: “What can you tell me about the weapon?

Ducky: “Over an inch wide, maybe a belt - there might be trace, I’ll know more when I get her home.

Gibbs: “Quickly Ducky - Tony, show Ruth the truck; McGee bring Abby, two minutes. (Gibbs follows walkway toward the building door; Tony, McGee and Ruth leave walking on the grass alongside the building, Palmer and Mallard stay with the body, Ducky watches Gibbs go, looks perplexed).

Scene 3

(in the parking garage, beside NCIS truck).

Abby and McGee arrive, join Tony and Ruth seen talking as we approach.

Tony: “Abby, meet Ruth, a nice jewish airforce Captain from Ireland - or maybe Idaho- she’s sitting Ziva, temporary detached, to us, from defense intelligence. Worked with Gibbs before.

Abby: “hi, anybody know what this is about?

Gibbs (entering from behind the truck) “it’s about our dead woman, she was coming to see me, said she wanted to turn herself in.”

Tony: “For?”

Gibbs: “Don’t know - but there’s something you need to know - and nobody else, clear?

(Tony, McGee, Abby, all looking at Ruth, who says nothing) “Yes boss, nobody else

Gibbs: It’s not over. somebody in NCIS is leaking information. Rivkin knew what we were doing before we did it - and not from Ziva. Last week an op none of you knew about went wrong - it was a test, Fornell set it up and a Chinese ship that should have gone north went west. Not coincidence.

Abby: “sooo, Ruth?”

Gibbs: “Yes, DIA outside review, I asked for Ruth - because it could have been one of us, maybe you Tony, or McGee.

Ruth: “The test was double blind. It’s not Gibbs either.”

Abby: “Of course not Gibbs, and not me - or Tony, or McGee - and not Ducky”

Gibbs: “Need to know, no slips: Ducky doesn’t have need to know. Nobody else. Clear?

This is guy is smart, he covers his tracks, thinks things through -but Margaret surprised him, this time he had no time, had to improvise, and he’s probably NCIS, one of us.

McGee, her office, find what she wanted to tell me - Ruth, Tony where she lived; Abby, Ducky might have weapon’s trace, he thinks a belt might have been used.

Scene 4

Ducky, doing autopsy, “a healthy young woman, took care of yourself didn’t you, my dear - I’m sorry I have to excise your skin like this, (cutting red skin from her neck) but it may help find who killed you - Mr. Palmer, did you know that “red neck” was used for over a hundred years to mean hard working farmers and only became a pejorative when a fellow named Joe Starobin, writing for the American Communist paper The Sunday Worker in 1947, used it in a memorable denunciation of FBI agent Robert Lamphere at the trial - for espionage- of one Mr. Eisler - yes, babyfaced, hardlipped, and the owner of a viciously flushing redneck, is how he described him - it just goes to show you how the press misconstructs the public’s image of who the good guys are.

Ducky: (Shakes head, finishes cutting in silence, turns body over, and triumphantly peels off the whole loop): “to Abby please, Mr. Palmer.”

(Palmer leaves with the skin in a jar, visibly relieved)

Ducky: “and now my dear, lets see what stuck where he struck”

Scene 5:

Bullpen

Gibbs: Tony?

Tony “got it boss, living above her means; place is neat, clean, clothes too expensive for her salary, Accounting student from Brigham Young, articling in NCIS internal audit, would have made CPA next year. No known boyfriend but takes the pill, and her bed got used.

Parents have been notified; mother’s here tomorrow, father’s a marine lieutenant Colonel doing his up and out working logistics support on Guam -that’s her connection here- he’ll get compassionate and be here day after tomorrow.

Ruth: “No outstanding debt, paid her student loans off last year, deposits, salary plus a regular $4,000 in cash every month since she started here; told the bank she sells cosmetics part time - nothing to support that - and no backtrack on the money. Rent is paid by post dated check - no negatives.

Gibbs: (to Tony) “Find him. (turns) McGee?

McGee: “Working on it boss, so far - her PC has software on it that’s not supposed to be there, patches external phone calls into our internal network. Looks like cell to skype - that’s a telephone thing for PCs, lets you make video calls. We block it at the firewall, but an internal origin is invisible. (His phone rings) Debug Logs are off, so don’t think we can trace where her calls went, but desktop logging was running on the server and it (phone rings) shows Skype running last at 3:12PM. Yesterday. Sorry, looks at phone, Abby’s got something (picks up phone); Hi Abbs?

McGee: Listens: “Way to go Abbs!, (hangs up, looks at Gibbs) Abby says there were 182 desktops running at the time of that last skype call, from server logs. We keep a month of boot records, and should be able to narrow the list by dropping those not running when her skype program was active over that period.

Vance (on walkway, above): “Gibbs, you catch your parking lot killer yet?”

Gibbs: “Working on it”

Vance “Ziva still in Israel?” (looks meaningfully at Gibbs, walks away)

(The PC on Ziva’s desk suddenly beeps, screen flashes)

Tony: (looks at Gibbs, nods) On it, boss

Ruth (pointing at PC), “How’s it doing that, I didn’t turn it on?”

Gibbs: McGee?

McGee: Remote on, it’s mainly used at night: the server turns it on to backup the harddisk, turns it off to save power. It’s beeping now for priority IM: hey, it’s from Director Vance - says “Charles Sonderstrom”.

Why send that? he was just here, oh oh, wait, that’s not his PC address, it’s not from him, I’ll go check the server logs to see if I can find out where it came from.

Gibbs: Go. Tony, Ruth?

Tony: On it boss

Ruth: (sitting down at Ziva’s desk and pushing the PC to the edge before picking up the phone) “You guys really have to use this stuff? I haven’t, not since 95 or something; McGee could you show me how to get the browser and email working on this?

McGee: “The group servers are in the basement, I have to go - but here, let me show you (hunches down so he can reach her keyboard) office login is

Tony: “Ziva whined and qvetched about it all the time too; you guys at DIA, use Macs the way Mossad does?

Ruth: (Paying attention to McGee, and thus distracted) “I don’t know what we use, Jerry calls them displays or something, from Sun or some name like that, I think.

McGee: (Standing up) “ok? I’ll go over it in more detail with you later, gotta go, the boss wants answers - Abby, then downstairs (Leaves)

Scene 6

Gibbs and Tony are in what looks like a US Army motor pool assembly and maintenance facility. A lieutenant too old for the job stands off to one side. Gibbs is talking to a technical Sargent whose chest tag says “Sonderstrom”.

Gibbs: “What were you doing in the naval yard at 8:30 this morning?

Charles: “Reporting to orders sir, or trying to - electronic work order this morning, yesterday’s datum, report to Lt. Commander Taggart on USS Monmouth instanter. Lt. Ferguson (pointing) has the order, he wasn’t in yet when I saw it, left it for him, burnt feet getting there before 10:30. Not in the yard at 8:30, got there about 10, sir.

But Commander Taggart wasn’t there sir, and his XO denied the order, - sent me back here.

Ferguson: “That’s right sir, this (waving a work order) was time stamped here at 8:52AM, today. I got in about 9 - found it on my desk.

Gibbs: “So what time did you get to the yard?

Charles: “I checked in at the gate sir, just after 10, I think. In both directions sir, I left the yard about forty minutes later. They’ll have the records, sir.

Gibbs: (to Ferguson) “This has yesterday’s action date - how could it not get to you until this morning?

Ferguson: “I don’t know sir, our system usually works well - and I know Sargent Sonderstrom was here yesterday, he was working on armor attachment for that two four (pointing at 6 wheel, 5 ton truck).

Gibbs: “Sargent, how well did you know Margaret Anders?

Charles: “Marge? What’s she got to do with this?

Gibbs: “how well did you know her?

Charles: “She’s a neighbor at home, in my classes at Clark Fork JS, haven’t seen her since joining up - more than five years now.

Gibbs: “So you didn’t go to see her at the yard?

Charles: “At the yard sir? What’s she doing there?

Tony: “Some one killed her, bashed her brains out in the parking lot

Charles: “Oh my god. (pause) Marge?. (Looks around vaguely, sits)

Gibbs: “Sargent: you went from the gate to the Monmouth, directly, is that right?

Charles: (somewhat weakly): “ah, no sir, I got a bit lost, parked at the wrong end, and went toward the towers at first, then got a ride most of the way back to the Monmouth with a lifter.

Gibbs: “Did you get his name?

Charles: “No Sir, big man, black; old hand with the machine. Why was Marge there?

Gibbs: “She worked for us. We’ll probably need to speak to you again - Mr. Ferguson, get Tony with your tech people: see if work orders with later dates came through the queue before this one - and check the alibi - cameras, gate MPs, find that driver. Find someone who saw him here this morning.

Tony “On it, boss.”

Scene 7

Things are quiet, McGee is helping Ruth learn to use her PC - they’re at Ziva’s desk.

McGee: “Then you just login here

Ruth: “What, Again? we already logged in - twice, I think.

McGee: (patience obviously fading a bit) “that was for the network, and the email server, this is the maps server

Ruth: “And I can’t keep my office docs open while doing this?

McGee: “No no, normally you can, this maps server is the exception, because once you log in it uses the office graphics interface to put labels and handles on the maps so you can manipulate them and if you’ve got docs open when it grabs it, something’s likely to crash and burn, so just close them first, ok?

Ruth: (sighs) “ok (phone rings, she picks it up): “O’Brien?”, “oh, hi Jerry” (listens) I’ll ask you to talk to Special Agent McGee - he speaks your language. (Looks at McGee) “This is is Jerry Malins, runs our systems, he has an idea about tracing those skype calls to a desktop. (holds out phone)

McGee: (horrifed)” You mean you talked about a security matter to a help desk guy?

Ruth (hand over phone): “help Desk? we don’t have anything like that, Jerry does everything, and he’s cleared for everything.

McGee: “So he’s head of IT for you guys?

Ruth: “Not all of DIA, no, he reports to somebody in Fort Bragg - just for us.

McGee: “That’s what? about five people?

Ruth: “About 400 I think

McGee: “One guy, 400 people, sure, and you don’t know how to use this stuff, what do you guys do, share an abacus? (Takes the phone) McGee: (Listens)

In the background Gibbs and Tony come in.

McGee: “Yes, IOS (listens) isn’t the default 36 hours? I’m sure no one would have changed it - should work, great idea! I’m on it, right now, Thanks! (hands phone back).

Ruth: “Good news?

McGee: “Maybe, skype uses its own port and the main switch downstairs will have built up a routing table for it - if I can dump that and see expiry times it’ll show exactly which PC the signals were routed to. (starts off toward the elevators)

Ruth: “McGee: I asked, Jerry said they’re Sun Rays - single signon, and everything works, not (leaning back in the chair to swing her feet onto the desk) like this shh (we know what she’s saying, but her feet hit the PC and we don’t hear the word over the sound of the thing crashing to the floor.)

Gibbs (from behind her): I think we’ve all wanted to do that a few times.

End of part one: Fade to “TBC” -and a teaser for next season:

Scene 8

The basement conference room - beside the NCIS truck.

Present are Tony, Ruth, Abby, Gibbs, and McGee

Gibbs: McGee?

McGee: “It’s Director Vance’s office. The times match up only for redirection to his laptop.

Tony: “Sonderstrom’s orders were dated yesterday, but came to the workflow machine this morning - after the murder - but nobody knows how.

Gibbs: “Could he have sent them himself, to explain his presence here?

Tony: “The Monmouth confirms his story, the gate times check out, the lift operator gets off at 11, confirmed the story by phone, will come in tomorrow. Not coincidence, but maybe somebody trailing a red herring for us?

Gibbs: Abby?

Abby: “there’s no usable weapon’s trace on the skin - just some polyethylene. If a belt was used, it got wrapped in plastic first. Ducky thinks the blow to her head came from something heavy, metal, and rectangular - like a very heavy buckle on a belt used as a whip - or a gun butt, probably wrapped in plastic too.

…

May 2nd, 2009

Oracle Office, MySQL, and other dreams

Posted by Paul Murphy @ 12:15 am

Categories: Applications, Database Management, Enterprise Policy, General, Linux, Software Licensing, Strategy, Sun

Tags: Oracle Corp., MySQL, Sun Microsystems Inc., OpenDocument Format, OpenDocument Format (ODF), StarOffice, Open Source, OpenOffice, Databases, Emerging Technologies

Most people don’t know this, but at one time Oracle Corporation ran an extensive internal beta adding a pretty good word processor and spreadsheet to the Oracle Office communications product most recently renamed Oracle Beehive and now positioned against both Exchange and Domino.

At the time I didn’t see the combination as competitive with Applixware or even Q-Office, but recommended it for sale because it did one thing better than any competitive product I knew of: it stored everything as rows in the standard database.

Sun does not own OpenOffice.org and Oracle, accordingly, isn’t buying it - but because Sun does own the StarOffice product and is the primary contributor to OpenOffice.org, Oracle is buying both influence with, and responsibility to, that community.

If Sun and Oracle now recreate the full Oracle Office by combining Beehive with StarOffice, ensuring standardized database storage for all data, and open sourcing some of Oracle’s forms based development tools into the bargain, almost everyone stands to gain.

The obvious effect, of course, is to give OpenOffice a tremendous technical advantage over Microsoft Office while, at the same time, adding communications technologies and a real Access competitor.

I think, however, that two rather more subtle effects would dominate the future impact such a product would have.

First there’s the impact on the whole ODF scene. Right now, and for the past ten years, I’d argue that the eminently logical separation of storage from format is what’s enabled Microsoft to stall on ODF by keeping the brightly colored ball bouncing in the public eye while quietly and effectively selling internal integration as the justification for knee capping ODF at every opportunity.

Force content and format information to be stored in a single, consistent, way however and Microsoft’s wiggle room get reduced while accurate format conversions become easier - meaning that a new Oracle Office would ultimately have an enormous impact directly in terms of getting truly open document format standards widely accepted and used - and indirectly in terms of empowering the Linux desktop.

Second, for this to really work in light of Sun’s existing licensing commitments, Oracle would have to open source the database and communications components for Oracle Office. Since MySQL is well suited to the job and already open source, my guess is that the pros and cons of using it would then tilt in favor - meaning that my hypothetical Oracle Office would boost the MySQL community first by creating long term support commitments and secondly by putting it at the core of a lot of Exchange replacements.

One note:

1. I’m assuming that Oracle will neither want, nor be able, to sell Sun’s MySQL asset as a going concern.

   The reason I’m making this assumption is that Sun’s billion dollars didn’t so much buy an asset as deny IBM the chance to increase Sun’s dependence on Oracle for hardware sales - meaning that there isn’t much there, and simply spinning the commitment off into a tax exempt foundation would return far less to Oracle shareholders than using MySQL to gain credibility and support in the open source community will.