October 21st, 2009

Gartner Magic Quadrant lawsuit: Sour grapes or real gripes?

Posted by Michael Krigsman @ 9:48 am

Categories: CIO issues, Project strategy, Research and statistics, Vendor relationships

Tags: Lawsuit, Quadrant, Analyst, Gartner Inc., Michael Krigsman

Industry analyst firm, Gartner, is the target of a lawsuit from software vendor, ZL Technologies, challenging the “legitimacy” of Gartner’s Magic Quadrant rating system. The suit has brought forth an array of divergent opinions.

Background. As one of the top analyst firms, with revenue in excess of a billion dollars, Gartner’s opinions and recommendations carry substantial weight with technology buyers and influencers.

On a special website page devoted to the lawsuit, ZL Technologies claims that Gartner’s Magic Quadrant does not present a fair and accurate portrayal of the software market. The company says:

Gartner’s use of their proprietary “Magic Quadrant” is misleading and favors large vendors with large sales and marketing budgets over smaller innovators such as ZL that have developed higher performing products.

Here is the original legal filing:

Read the rest of this entry »

September 24th, 2009

Pay to play: The vendor / analyst mating dance

Posted by Michael Krigsman @ 10:12 am

Categories: CIO issues, Consulting, Devil's Triangle, IT issues, Vendor relationships

Tags: Analyst, Games, Enterprise Software, Personal Technology, Software, Michael Krigsman

Certain elements of the enterprise software ecosystem contribute directly to failed IT projects. For example, I have written extensively about the Devil’s Triangle, which describes conflicting relationships among enterprise buyers, technology vendors, and system integrators.

• Related: Exploring the Devil’s Triangle

Industry analysts can also contribute to failures by creating mismatched expectations between customers and vendors. This happens when analysts slant coverage toward a specific vendor in hopes of later securing a lucrative consulting contract or retainer.

I put together a diagram to illustrate:

CMS Watch founder, Tony Byrne, details the process in a blog post titled The vendor-analyst echo chamber game. Here’s a quote:

[S]oftware vendors typically don’t spend money with analyst firms to bribe them outright. Rather, they purchase attention through which they can try to get an analyst to define the marketplace and customer challenges according to that particular vendor’s approach.

It’s the vendor-analyst echo chamber game, designed to manufacture artificial demand.

Although the diagram describes a fundamental conflict of interest, many analysts have integrity beyond reproach. The best analysts play an important role in helping buyers make wise strategic decisions. Those excellent folks provide substantial value and contribute to project success by aligning expectations between technology buyers and sellers.

What do you think about this issue? Please share your thoughts.

June 12th, 2009

TechCrunch Research: Big deal or not?

Posted by Michael Krigsman @ 6:22 am

Categories: Consulting, Uncategorized, Vendor relationships

Tags: TechCrunch, Advisor, PROJECT FAILURES ANALYSIS TechCrunch, TechCrunch Research, Investment, Finance, Michael Krigsman

TechCrunch, the opinionated startup-focused blog network, has launched a low-cost analytic research service. Given the company’s high profile, this research offering cannot be ignored. However, does this new TechCrunch initiative provide value to buyers or threaten established analyst firms?

TechCrunch describes its 1Q 2009 Report report this way:

[It] provides key take aways and statistical support for the major trends of Q1. The report covers trends in start-up foundings, products, financings and exits across a variety of technology sectors: consumer media and entertainment, social networking, cloud computing, mobile communications, search, advertising and ecommerce, consumer electronics and clean tech.

The report costs $149 and partially overlaps the outstanding, web-based MoneyTree Report, which is free and has been around for years. PriceWaterhouseCoopers and the National Venture Capital Association sponsor MoneyTree.

THE PROJECT FAILURES ANALYSIS

TechCrunch keeps its finger on the pulse of startups, writing with a colorful (some say arrogant) tone and style. This formula has propelled TechCrunch’s popularity and influence among those following technology startups and investment.

Read the rest of this entry »

August 22nd, 2008

'Debunking IT Project Failure Myths' [podcast]

Posted by Michael Krigsman @ 11:01 am

Categories: CIO issues, Cultural issues, IT issues, Interview, Podcast, Project portfolio management, Project strategy

Tags: Information Technology, Exec, Strategy, Management, Michael Krigsman

According to various studies, at least 30% of all IT projects fail in some important way. Failure rates seem to have plateaued at this level because most organizations don’t really understand why their IT projects go down. As a result, failures persist, with some organizations even proclaiming, “We’re not at fault because we did everything right.” Such attitudes are misguided.

In his recent report, titled “Debunking IT Project Failure Myths,” Lewis Cardin, a former CIO and currently senior analyst at Forrester Research, states:

Firms commonly use three metrics to decide whether a project effort is successful: Did the project meet its schedule, stay within budget, and deliver on requirements?… Firms use these same measurements to establish project success rates simply because they are so obvious and business execs easily understand them. IT execs often get measured on these outcomes and are held accountable for the results delivered by project managers; when a project is deemed a failure, this accountability can be bad news for IT. The problem is that this IT accountability is frequently misplaced. Worse still, the conclusion of failure is often incorrect.

Translation from the trenches: the real sources of IT failure lie in issues like project management culture, which neither IT nor most business people are comfortable analyzing, let alone fixing. Lewis’ research describes four critical dimensions that project stakeholders frequently misdiagnose, resulting in repeated failure:

Unresponsive governance, which leaves project decisions hanging. IT project governance has the role of project approval, problem resolution, direction-setting, and communication with business stakeholders…. Unless the cause of this delay is visible and shown to be for governance reasons, someone is going to point fingers — months later — to defective project management and not the real source of the delay.

Lack of communication from change management, which leads to false conclusions. Business managers may change requirements during project execution…. At project completion, the business has forgotten about the improved value component while memories are crystal clear about the increased dollar and time investment. A project has a high probability of being tallied under the failure column when in fact it may have been a noteworthy success.

An unrealistic project plan, which dooms the best project. All too often, when these projects go on the rails of the original project plan, PMs must spend more time on damage control with steering committees and project resources rather than on execution — doubling their work when it is least desirable to do so.

First-number syndrome, which makes business execs forget it’s an estimate. When projects are first sized, which is likely to occur before they are approved, estimates of cost, time, and resources are preliminary with a wide confidence interval…. But business execs remember the number and forget how uncertain it is…[and] may see this simply as increasing costs, not as the inevitable result of greater knowledge.

Issues such as executive sponsorship, business case, usability, and vendor integrity play a large role in determining the outcome of IT deployments. Unfortunately, most organizations don’t pay sufficient attention to these key areas because they’re hard to measure. Nonetheless, IT project success is not possible without paying careful attention to the real causes of failure.

ABOUT THE PODCAST

I urge you to spend 12 minutes and listen to the attached podcast. Lewis and I explore these issues during a provocative and informing conversation. The discussion of change management and the cultural determinants of IT failure alone is worth your time.

April 7th, 2008

IT failure at Heathrow T5: What really happened

Posted by Michael Krigsman @ 7:48 pm

Categories: CIO issues, End-user impact, Government projects, IT issues, Implementation, Project failures, Project management

Tags: Software, CIO, Baggage, British Airways Plc., Information Technology, Problem, T5, Terminal 5, BA, Vanderlande Industries

Anyone having the misfortune to pass recently through Heathrow’s new Terminal 5 knows the huge project debuted as a case study in full-blown systems failure. Most reports have focused on stranded customers, a crazed supermodel, and Queen Elizabeth’s grand opening speech, which called the terminal “a 21st Century gateway to Britain.”

Here’s my rundown of the major failures and why this debacle happened.

Project scope. Terminal 5 represents an enormous investment in buildings, systems, technology, systems, and human processes. From ComputerworldUK:

Terminal 5 has cost BA and the British Airports Authority £4.3bn [$8.5 billion] to build and outfit. BA says around £75m of these costs are for technology, while BAA invested at least another £175m in IT systems.

The work has involved 180 IT suppliers and seen 163 IT systems installed.

The sheer physical size of the project is impressive. From the official brochure:

Terminal 5’s technical complexity matches its physical size. According to the British Airports Authority (BAA), “It has taken 400,000 man-hours of software engineering just to develop the complex system, and coding work is set to continue even after the initial installation begins.” From CIO UK:

The project is a complex one. T5 will involve 180 IT suppliers and run 163 IT systems, 546 interfaces, more than 9,000 connected devices, 2,100 PCs and “enough cable to lay to Istanbul and back”. It will contain 175 lifts, 131 escalators and 18km of conveyor belts for baggage handling. [British Airways CIO Paul] Coby says that even the construction of T5 involves: “creating a small town with a full telecommunications network for the construction workers, merely to enable the terminal to be built.”

Political fallout. The impact of the problem has hit the highest levels of British politics. The Guardian reports:

Foreign secretary David Miliband also joined the criticism of BA as it emerged that the crisis could cost the airline £50m. Miliband said a furious EU ministerial counterpart, whose bags had been lost at T5, had asked him to reprimand BA and BAA. Writing on his blog, the foreign secretary said he had been harangued at a gathering of EU foreign ministers, days after the terminal’s opening was marred by a botched baggage handling operation and multiple flight cancellations.

UK Under-Secretary of State for Transport, Jim Fitzpatrick, testified before the House of Commons under harsh questioning:

The management of terminal 5 is an operational matter for British Airways and BAA, but that does not mean that the Government are not taking a keen interest in seeing that the difficulties T5 has faced since it opened last week are addressed and resolved as quickly as possible. On it first day in operation, T5’s bespoke baggage system was affected by a number of issues. First, there were technical software problems, but more significantly BA’s challenge was integrating teams of staff, and it has been addressing this as a priority….

[N]ational pride has been dented. Her Majesty opened T5 to national fanfare and delight in early March. The following week, when the A380 arrived for its first flight, many of us believed that Heathrow had turned a corner and that the bad publicity of recent years would turn into positive publicity, notwithstanding the importance of scrutiny as regards the expansion. Clearly, that was not the case.

Baggage systems failure. Stranded travelers immediately understood how problems with the baggage handling system affected them personally. A passenger told the BBC:

“Apparently the computer software told the baggage people that the flight had taken off. So everyone in the plane just watched as all their suitcases were taken back into the terminal instead of being loaded on.”

One baggage worker told the BBC the situation was “mayhem” and that the technical problems had been known about for some time. “They have been doing tests on the belt system for the last few weeks and knew it wasn’t going right. The computer cannot cope with the number of bags going through.”

Airport-technology.com describes the system:

The baggage handling system at T5 will be the largest baggage handling system in Europe for a single terminal. There will be two systems including a main baggage sorter and a fast track system.

The system was designed by an integrated team from BAA, BA and Vanderlande Industries of the Netherlands, and will handle both intra-terminal and inter-terminal luggage and will actually process 70,000 bags a day.

Bags undergo several processes on the way through the system including automatic identification, explosives screening, fast tracking for urgent bags, sorting and automatic sorting and passenger reconciliation.

Vanderlande adds technical detail:

The baggage handling system has a total of 12 transfer break lines and 132 check-in desks according to the ‘walk through’ concept for easy flow of large number of passengers. Bags are automatically read, screened and sorted to their final build location via two electrical HELIXORTERS™. Early bags are stored in a BAGSTORE™ warehouse with 4,000 positions, allowing individual access to each bag any time. Late bags can be transported at high speed via BAGTRAX™ directly to the head of stands where the awaiting aircraft are parked.

Vanderlande Industries forms part of the T5 Strategic Team and is supported by BAA’s Baggage Delivery Team. Vanderlande Industries is fully responsible for design, software development, manufacturing, installation, commissioning and integration of this baggage handling system.

Computer Weekly reports that IBM supplied software components:

The system was designed with Dutch company Vanderlande and IBM. The IBM software works out where the bags are supposed to be going, and logistics software works out the best way to get there.

Car park problems. Among all the other problems, the BBC reports that terminal staff had trouble new parking garages, which exacerbated the other issues. Given everything else, this one’s just icing on the cake.

Privacy concerns. The British Airports Authority eliminated biometric security measures (fingerprinting all passengers) immediately prior to the terminal opening due to legal questions. Computer Weekly reports:

Heathrow Airport owner BAA is pulling a biometric fingerprint system at the new Terminal 5 the day before the building opens to the public, after the Information Commissioner’s Office (ICO) raised concerns about the system.

The system would have taken the fingerprints of all domestic passengers as they passed through the security gates, and was intended to prevent illegal immigration.

Computing elaborated on this issue:

The Information Commissioner’s Office (ICO) expressed concern that the move may breach data protection laws – even though BAA planned to delete fingerprint data after 24 hours.

“We have concerns about the routine collection of fingerprint information from passengers and we will require reassurance from BAA that the data protection implications of the proposals have been fully addressed,” said an ICO spokeswoman.

BAA will now use a single digital photograph to confirm passengers at the gate are the same as those that check in – a system used in most other international UK airports.

THE PROJECT FAILURES ANALYSIS

Canceled flights, lost baggage, and substantial delays were symptoms that arose from failures in management, planning, and testing on this expensive and complex project.

As reported in ComputerworldUK, the T5 systems represented a massive business transformation effort for British Airways:

The technology has let British Airways create what is believed to be one of the airline industry’s first Order Data Stores (ODS). The ODS will combine customer information from across the complete booking-to-fulfillment lifecycle in a single unified data layer.

The software means British Airways can now implement new business processes that bridge disparate global distribution systems and passenger service systems, enabling it to action sophisticated workflows against end-to-end customer orders. BA said the capability is facilitating “the implementation of new business models and help[ing] the airline accelerate its transformation of the IT function into a business enabler to support key business goals and initiatives.”

British Airways CIO, Paul Coby, described the process changes the company is trying to incorporate into T5. From CIO UK:

Originally designed for the automotive industry, the lean model is popular in manufacturing circles as a process management philosophy that focuses on standardising processes in order to reduce costs and time waste and improve efficiency. T5, according to Coby, has many qualities that are similar to a factory, in that it aims to eliminate waste, simplify and streamline processes, create flow and create a culture of continuous improvement.

BA uses the lean model to assist the carrier to reduce overheads and manage the enormous task of handling business change and the massive integration challenge involved in this large-scale project. Coby explains that lean involves using simple, repeatable processes and only adopting complex technology when there is no alternative.

Training issues. Given the new system’s complexity, staff training and preparation was clearly lacking. BA acknowledged the critical importance of training in the kind of business transformation program that T5 represents. From Computing:

Training and familiarisation procedures were carried out for a year so that staff could test the new IT systems in place throughout the building.

“Delivering more than 200 IT projects was easy when compared to the people element of the plan,” said BA’s programme head of information management for T5, Glenn Morgan.

“Getting people to understand the new role of technology and buy into the new proposition was our biggest challenge,” he said. “For some projects, people were asking for new systems to support their work, but we actually changed the process and brought no IT.”

British Airways underscored the impact of insufficient training. From the BBC:

BA blamed the glitches on problems with “staff familiarisation”, which had a knock-on effect on bags and flights. A BA spokeswoman said delays in getting staff screened for work and other technical difficulties had caused the airline problems.

Planning, testing, and software quality assurance (QA). British Airways started system testing a year ago. From CIO UK:

BA’s CIO, Paul Coby told CIO UK [in March 2007] the IT work to support such a large-scale, new-build project was also going well. “Devices are deployed, connections are being integrated and 2007 will be testing year. The airline is moving onto the T5 systems, so they run for a year ready to operate at the new terminal when it opens in 2008. This is the year we put the IT infrastructure and systems in place to manage our people and passengers,” he said.

However, BA underestimated the testing, integration, and release planning required to achieve a trouble-free launch. CIO UK described BA’s misplaced sense of invincibility:

Almost every new airport has opened late, cost more and had major operational issues, and, according to [Nick Gaines, director of business critical systems and IT at the British Airports Authority], “system integration problems” are often blamed. “This will not be the case for T5. BAA and BA plan working together for system integration over two years, we’ve been focused on bringing systems together. Integration is not just plugging technology together; it’s about people, processes and systems working together.”

In a moment of understatement adding insult to injury, British Airways CEO, Willie Walsh, commented:

We are working hard to tackle the difficulties we have had with the terminal’s baggage system. From time to time problems have developed that were not encountered during the extensive trials.

The systems incorporated in T5 severely taxed BA’s planning, testing, and deployment capabilities. In an irony perhaps better suited to a BBC sitcom, the British Airports Authority recently solicited for software quality assurance vendors. In February, 2008 ComputerworldUK reported:

Airports authority BAA has begun looking for a service provider to work with it to help develop consistent IT systems at Heathrow and across the authority’s airports. It has put out a tender for a framework agreement to find suppliers able to risk-assess any new systems it develops – and to monitor and maintain those same systems once they are in use.

The airports authority wants any suppliers to maintain software quality by putting in place rigorous development processes and testing procedures prior to systems go-live.

And it said it also wants firms to “conduct risk and integrity analyses of proposed system architectures and provide guidance on assurance issues.”

My take: there’s never enough time or budget to test, train, and prepare properly for launch. But after the deluge, when problems have been exposed and the bigwigs embarrassed by failure, time and money magically appear. Geez, isn’t that always what seems to happen?

March 9th, 2008

Sleazy analysts and corporate weasels

Posted by Michael Krigsman @ 10:02 pm

Categories: CIO issues, Consulting, Vendor relationships

Tags: Analyst, Blog, Business Ethics, Blogging, Leadership, Management, Internet, Michael Krigsman

IT buyers look to industry analysts for wisdom in understanding the past, counsel in handling the present, and advice when future plans go awry. Unfortunately, this symbiotic relationship between analysts and their corporate masters can include a complicated mix of mutual respect and fear, punctuated by greed and need on both sides.

The Institute of Industry Analyst Relations recently blogged about analyst ethics:

Any analyst firm which values its long-term reputation in the market has to ensure that its research is independent….

However we do need to be realistic about the economics of the analyst business. Most analyst firms couldn’t exist without vendor cash - be it via sponsored research, consulting projects or speaking engagements.

And so long as analyst firms clearly communicate who is sponsoring their work, I’m fine with that.

The blog raises questionable analyst practices such as the following:

• the analyst that writes blog posts promoting a project that his consultancy is involved in - without disclosing his connection
• the division of a large group that prioritizes briefings based on the likelihood of selling reprints of the resulting company profile
• the analysts that use a briefing as an opportunity to pitch their own services

THE PROJECT FAILURES ANALYSIS

Analysts can achieve transparency, and neutralize many ethics violations and conflicts of interest, by fully and completely disclosing their affiliations. Unfortunately, such disclosure is not always forthcoming, and some analysts use lack of transparency as a tool to mislead clients and the unsuspecting public with so-called “objective” research that has in fact been compromised.

Ethical lapses also occur among the corporations that engage analysts, some of whom use their buying power as a lever to manipulate undue influence over research and reports. Using the promise of large fees and ongoing retainers, these corporate weasels subtly encourage analysts to bias their recommendations to the sponsor’s advantage. Again, unsuspecting readers are wrongly led to believe the analyst’s report and conclusions are unbiased.

The best analysts skillfully manage these conflicts and tensions without compromising either their integrity or their work. For example, Jon Collins, of Freeform Dynamics, told me that analysts who pitch their own services at briefings are “unsavory and unproductive.” He wrote that’s it’s “difficult to be both transparent and unethical.” Similarly, James Governor, of Redmonk, blogged “You Can Buy Our Thinking But You Can’t Buy Our Opinion.”

The bottom line: When talking with analysts, remain alert to potential conflicts of interest and ethical gaps. Ask about their affiliations, demand full disclosure, and walk away if you don’t get satisfactory answers. The best analysts will welcome your questions and withstand your scrutiny; the others aren’t worth your time or money.

Update 3/10/08: Changed the title to better reflect the content.

January 3rd, 2007

Vendor Integrity

Posted by Michael Krigsman @ 7:39 pm

Categories: Vendor relationships

Tags: Hiring, Lawyer, Dennis Howlett, Michael Krigsman

Dennis Howlett raises some questions to be used when evaluating lawyers and accountants. Similar guidelines could be applied when hiring consultants on IT projects.

• How messy is your lawyer’s desk? When they bill you for thirty minutes of “file review,” how much of that time was spent looking for your file?


• When you call a lawyer for the first time, how long does it take for him to return your calls? After you hire that lawyer, expect it to take at least three times as long. Same goes for e-mails.


• Does your lawyer have reputation for being a “bulldog?” That probably means they are an asshole. To everyone.

Want your IT project to be successful? Find a really well-qualified vendor who does the right thing, puts your needs before his personal agendas, and is honest. When you find that person, don’t nickel and dime him, pay him properly for his work, and keep hiring him back.

—–