November 18th, 2009

IBM feels cozy on sidelines as Oracle-Sun deal languishes in anti-trust purgatory

Posted by Dana Gardner @ 3:44 am

Categories: Apache, BI, Cloud computing, Eclipse, Enterprise Java, HP, Hardware Infrastructure, IBM, Intellectual Property, Java, Microsoft, Open Source, Oracle, SAP, SOA, Silicon Valley, Software Development, Software Infrastructure, Sybase, System Z, Wall Street, business intelligence, database, datacenters, mainframe

Tags: Oracle Corp., Antitrust, Sun Microsystems Inc., Steve Mills, IBM Corp., Programming Languages, Open Source, Java, Databases, Software Development

You have to know when to hold them, and when to fold them. That’s the not just slightly smug assessment by IBM executives as they reflect — with twinkles in their eyes — on the months-stalled Oracle acquisition of Sun Microsystems, a deal that IBM initially sought but then declined earlier this year.

Chatting over drinks at the end of day one of the Software Analyst Connect 2009 conference in Stamford, Conn., IBM Senior Vice President and IBM Software Group Executive Steve Mills told me last night he thinks the Oracle-Sun deal will go through, but it won’t necessarily be worth $9.50 a share to Oracle when it does.

“He (Oracle Chairman Larry Ellison) didn’t understand the hardware business. It’s a very different business from software,” said Mills.

Mills seemed very much at ease with IBM’s late-date jilt of Sun (Sun was apparently playing hard to get in order to get more than $9.40/share from Big Blue’s coffers). IBM’s stock price these days is homing in on $130, quite a nice turn of events given the global economy.

Sun is trading at $8.70, a significant discount to Oracle’s $9.50 bid, reflecting investor worries about the fate of the deal now under scrutiny by European regulators, Mill’s views notwithstanding.

IBM Software Group Vice President of Emerging Technology Rod Smith noted the irony — perhaps ancient Greek tragedy-caliber irony — that a low market share open source product is holding up the biggest commercial transaction of Sun’s history. “That open source stuff is tricky on who actually makes money and how much,” Smith chorused.

Should Mills’s prediction that Oracle successfully maintains its bid for Sun prove incorrect, it could mean bankruptcy for Sun. And that may mean many of Sun’s considerable intellectual property assets would go at fire-sale prices to … perhaps a few piecemeal bidders, including IBM. Smith just smiled, easily shrugging off the chill (socks in tact) from the towering “IBM” logo ice sculpture a few steps away.

And wouldn’t this hold up go away if Sun and/or Oracle jettisoned MySQL? Is it pride or hubris that makes a deal sour for one mere grape? Was the deal (and $7.4 billion) all about MySQL? Hardly.

Many observers think that Sun’s Java technology — and not its MySQL open source database franchise — should be of primary concern to European (and U.S.) anti-trust mandarins. I have to agree. But Mills isn’t too concerned with Oracle’s probable iron-grip on Java …, err licensing. IBM has a long-term license on the technology, the renewal of which is many years out. “We have plenty of time,” said Mills.

Yes, plenty of time to make Apache Harmony a Java doppelganger — not to mention the Java market-soothing effects of OSGi and Eclipse RCP. [Hey, IBM invented Java for the server for Sun, it can re-invent it for something else ... SAP?]

Unlike some software titans, Mills is clearly not living in a “reality distortion field” when it comes to Oracle’s situation.

“We’re in this for the long haul,” said Mills, noting that he and IBM have have been competing with Oracle since August 1993 when IBM launched its distributed DB2 product. “All of our market share comes at the expense of Oracle’s,” said Mills. “And we love to do benchmarks again Oracle.”

Even as the Fates seem to be on IBM’s side nowadays, the stakes remain high for the users of these high-end database technologies and products. It’s my contention that we’re only now entering the true data-driven decade. And all that data needs to run somewhere. And it’s not going to be in MySQL, no matter who ends up owning it.

November 16th, 2009

ZapThink explores the four stages of SOA governance that lead to business agility

Posted by Dana Gardner @ 7:09 am

Categories: Agile Development, Application Lifecycle Management, Cloud computing, HP, IBM, IT Management, IT Service Management, ITIL, Microsoft, Progress Software, SOA, SOA Governance, SOA architect, Software Development, Web Services, database, datacenters, governance, management

Tags: ZapThink LLC, Policy, SOA, SOA Governance, Service-Oriented Architecture (SOA), Web Services, Middleware, Enterprise Software, Software, Dana Gardner

This guest post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

By Jason Bloomberg

For several years now, ZapThink has spoken about SOA governance “in the narrow” vs. SOA governance” in the broad.” SOA governance in the narrow refers to governance of the SOA initiative, and focuses primarily on the service lifecycle.

When vendors try to sell you SOA governance gear, they’re typically talking about SOA governance in the narrow. SOA governance in the broad, in contrast, refers to IT governance in the SOA context. In other words, how will SOA help with IT governance (and by extension, corporate governance) once your SOA initiative is up and running?

In both our Licensed ZapThink Architect Boot Camp as well as our newer SOA and Cloud Governance Course, we also point out how governance typically involves human communication-centric activities like architecture reviews, human management, and people deciding to comply with policies. We point out this human context for governance to contrast it to the technology context that inevitably becomes the focus of SOA governance in the narrow. There is an important technology-centric SOA governance story to be told, of course, as long as it’s placed into the greater governance context.

One question we haven’t yet addressed in depth, however, is how these two contrasts — narrow vs. broad, human vs. technology — fit together. Taking a closer look, there’s an important trend taking shape, as organizations mature their approach to SOA governance, and with it, the overall SOA effort. Following this trend to its natural conclusion highlights some important facts about SOA, and can help organizations understand where they want to end up as their SOA initiative reaches its highest levels of maturity.

Introducing the SOA governance grid

Whenever faced with to orthogonal contrasts, the obvious thing to do is put them in a grid. Let’s see what we can learn from such a diagram:

First, let’s take a look at what each square contains, starting with the lower left corner and moving clockwise, because as we’ll see, that’s the sequence that corresponds best to increasing levels of SOA maturity.

1. Human-centric SOA governance in the narrow

As organizations first look at SOA and the governance challenge it presents, they must decide how they want to handle various governance issues. They must set up a SOA governance board or other committee to make broad SOA policy decisions. We also recommend setting up a SOA Center of Excellence to coordinate such policies across the whole enterprise.

These policy decisions initially focus on how to address business requirements, how to assemble and coordinate the SOA team, and what the team will need to do as they ramp up the SOA effort. The output of such SOA governance activities tend to be written documents and plenty of conversations and meetings.

The tools architects use for this stage are primarily communication-centric, namely word processors and portals and the like. But this stage is also when the repository comes into play as a place to put many such design time artifacts, and also where architects configure design time workflows for the SOA team. Technology, however, plays only a supporting role in this stage.

2. Technology-centric SOA governance in the narrow

As the SOA effort ramps up, the focus naturally shifts to technology. Governance activities center on the registry/repository and the rest of the SOA governance gear. Architects roll up their sleeves and hammer out technology-centric policies, preferably in an XML format that the gear can understand. Representing certain policies as metadata enables automated communication and enforcement of those policies, and also makes it more straightforward to change those policies over time.

This stage is also when run time SOA governance begins. Certain policies must be enforced at run time, either within the underlying runtime environment, in the management tool, or in the security infrastructure. At this point the SOA registry becomes a central governance tool, because it provides a single discovery point for run time policies. Tool-based interoperability also rises to the fore, as WS-I compliance, as well as compliance with the Governance Interoperability Framework or the CentraSite Community become essential governance policies.

3. Technology-centric SOA governance in the broad

The SOA implementation is up and running. There are a number of services in production, and their lifecycle is fully governed through hard work and proper architectural planning. Taking the SOA approach to responding to new business requirements is becoming the norm. So, when new requirements mean new policies, it’s possible to represent some of them as metadata as well, even though the policies aren’t specific to SOA.

Such policies are still technology-centric, for example, security policies or data governance policies or the like. Fortunately, the SOA governance infrastructure is up to the task of managing, communicating, and coordinating the enforcement of such policies. By leveraging SOA, it’s possible to centralize policy creation and communication, even for policies that aren’t SOA-specific.

Sometimes, in fact, new governance requirements can best be met with new services. For example, a new regulatory requirement might lead to a new message auditing policy. Why not build a service to take care of that? This example highlights what we mean by SOA governance in the broad. SOA is in place, so when a new governance requirement comes over the wall, we naturally leverage SOA to meet that requirement.

4. Human-centric SOA governance in the broad

This final stage is the most thought-provoking of all, because it represents the highest maturity level. How can SOA help with the human activities that form the larger picture of governance in the organization? Clearly, XML representations of technical policies aren’t the answer here. Rather, it’s how implementing SOA helps expand the governance role architecture plays in the organization. It’s a core best practice that architecture should drive IT governance. When the organization has adopted SOA, then SOA helps to inform best practices for IT governance overall.

The impact of SOA on enterprise architecture (EA) is also quite significant. Now that EAs increasingly realize that SOA is a style of EA, EA governance is becoming increasingly service-orientated in form as well. It is at this stage that part of the SOA governance value-proposition benefits the business directly, by formalizing how the enterprise represents capabilities consistent with the priorities of the organization.

The ZapThink take

The big win to moving to the fourth stage is in how leveraging SOA approaches to formalize EA governance impacts the organization’s business agility requirement. In some ways business agility is like any other business requirement, in that proper business analysis can delineate the requirement to the point that the technology team can deliver it, the quality team can test for it, and the infrastructure can enforce it. But as we’ve written before, as an emergent property of the implementation, business agility is a different sort of requirement from more traditional business requirements in a fundamental way.

A critical part of achieving this business agility over time is to break down the business agility requirement into a set of policies, and then establish, communicate, and enforce those policies — in other words, provide business agility governance. Only now, we’re not talking about technology at all. We’re talking about transforming how the organization leverages resources in a more agile manner by formalizing its approach to governance by following SOA best practices at the EA level. Organizations must understand the role SOA governance plays in achieving this long-term strategic vision for the enterprise.

This guest post comes courtesy of Jason Bloomberg, managing partner at ZapThink.

SPECIAL PARTNER OFFER

SOA and EA Training, Certification,
and Networking Events

In need of vendor-neutral, architect-level SOA and EA training? ZapThink’s Licensed ZapThink Architect (LZA) SOA Boot Camps provide four days of intense, hands-on architect-level SOA training and certification.

Advanced SOA architects might want to enroll in ZapThink’s SOA Governance and Security training and certification courses. Or, are you just looking to network with your peers, interact with experts and pundits, and schmooze on SOA after hours? Join us at an upcoming ZapForum event. Find out more and register for these events at http://www.zapthink.com/eventreg.html.

November 9th, 2009

Here's why text-based content access and management play crucial roles in real-time BI

Posted by Dana Gardner @ 2:27 pm

Categories: BI, Enterprise 2.0, Google, Government, HP, IBM, Podcasts, Software Development, Software Infrastructure, Web Services, business intelligence, content delivery network, database, management, search

Tags: Dana Gardner

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download a copy. Learn more. Sponsor: Kapow Technologies.

Text-based content and information from across the Web are growing in importance to businesses. The need to analyze web-based text in real-time is rising to where structured data was in importance just several years ago.

Indeed, for businesses looking to do even more commerce and community building across the Web, text access and analytics forms a new mother lode of valuable insights to mine.

As the recession forces the need to identify and evaluate new revenue sources, businesses need to capture such web data services for their business intelligence (BI) to work better, deeper, and faster.

In this podcast discussion, Part 3 of a series on web data services for BI, we discuss how an ecology of providers and a variety of content and data types come together in several use-case scenarios.

In Part 1 of our series we discussed how external data has grown in both volume and importance across the Internet, social networks, portals, and applications. In Part 2, we dug even deeper into how to make the most of web data services for BI, along with the need to share those web data services inferences quickly and easily.

Our panel now looks specifically at how near real-time text analytics fills out a framework of web data services that can form a whole greater than the sum of the parts, and this brings about a whole new generation of BI benefits and payoffs.

To help explain the benefits of text analytics and their context in web data services, we’re joined by Seth Grimes, principal consultant at Alta Plana Corp., and Stefan Andreasen, co-founder and chief technology officer at Kapow Technologies. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Grimes: “Noise free” is an interesting and difficult concept when you’re dealing with text, because text is just a form of human communication. Whether it’s written materials, or spoken materials that have been transcribed into text, human communications are incredibly chaotic … and they are full of “noise.” So really getting to something that’s noise-free is very ambitious.

… It’s become an imperative to try to deal with the great volume of text — the fire hose, as you said — of information that’s coming out. And, it’s coming out in many, many different languages, not just in English, but in other languages. It’s coming out 24 hours a day, 7 days a week — not only when your business analysts are working during your business day. People are posting stuff on the web at all hours. They are sending email at all hours.

If you want to keep up, if you want to do what business analysts have been referring to as a 360-degree analysis of information, you’ve got to have automated technologies to do it.

… There are hundreds of millions of people worldwide who are on the Internet, using email, and so on. There are probably even more people who are using cell phones, text messaging, and other forms of communication.

If you want to keep up, if you want to do what business analysts have been referring to as a 360-degree analysis of information, you’ve got to have automated technologies to do it. You simply can’t cope with the flood of information without them.

Fortunately, the software is now up to the job in the text analytics world. It’s up to the job of making sense of the huge flood of information from all kinds of diverse sources, high volume, 24 hours a day. We’re in a good place nowadays to try to make something of it with these technologies.

Andreasen: … There is also a huge amount of what I call “deep web,” very valuable information that you have to get to in some other way. That’s where we come in and allow you to build robots that can go to the deep web and extract information.

… Eliminating noise is getting rid of all this stuff around the article that is really irrelevant, so you get better results.

The other thing around noise-free is the structure. … The key here is to get noise-free data and to get full data. It’s not only to go to the deep web, but also get access to the data in a noise-free way, and in at least a semi-structured way, so that you can do better text analysis, because text analysis is extremely dependent on the quality of data.

Grimes: … [There are] many different use-cases for text analytics. This is not only on the Web, but within the enterprise as well, and crossing the boundary between the Web and the inside of the enterprise.

Those use-cases can be the early warning of a Swine flu epidemic or other medical issues. You can be sure that there is text analytics going on with Twitter and other instant messaging streams and forums to try to detect what’s going on.

… You also have brand and reputation management. If someone has started posting something very negative about your company or your products, then you want to detect that really quickly. You want early warning, so that you can react to it really quickly.

We have some great challenges out there, but . . . we have great technologies to respond to those challenges.

We have a great use case in the intelligence world. That’s one of the earliest adopters of text analytics technology. The idea is that if you are going to do something to prevent a terrorist attack, you need to detect and respond to the signals that are out there, that something is pending really quickly, and you have to have a high degree of certainty that you’re looking at the right thing and that you’re going to react appropriately.

… Text analytics actually predate BI. The basic approaches to analyzing textual sources were defined in the late ’50s. Actually, there is a paper from an IBM researcher from 1958, that defines BI as the analysis of textual sources.

…[Now] we want to take a subset of all of the information that’s out there in the so-called digital universe and bring in only what’s relevant to our business problems at hand. Having the infrastructure in place to do that is a very important aspect here.

Once we have that information in hand, we want to analyze it. We want to do what’s called information extraction, entity extraction. We want to identify the names of people, geographical location, companies, products, and so on. We want to look for pattern-based entities like dates, telephone numbers, addresses. And, we want to be able to extract that information from the textual sources.

Suitable technologies

All of this sounds very scientific and perhaps abstruse — and it is. But, the good message here is one that I have said already. There are now very good technologies that are suitable for use by business analysts, by people who aren’t wearing those white lab coats and all of that kind of stuff. The technologies that are available now focus on usability by people who have business problems to solve and who are not going to spend the time learning the complexities of the algorithms that underlie them.

Andreasen: … Any BI or any text analysis is no better than the data source behind it. There are four extremely important parameters for the data sources. One is that you have the right data sources.

There are so many examples of people making these kind of BI applications, text analytics applications, while settling for second-tier data sources, because they are the only ones they have. This is one area where Kapow Technologies comes in. We help you get exactly the right data sources you want.

The other thing that’s very important is that you have a full picture of the data. So, if you have data sources that are relevant from all kinds of verticals, all kinds of media, and so on, you really have to be sure you have a full coverage of data sources. Getting a full coverage of data sources is another thing that we help with.

Noise-free data

We already talked about the importance of noise-free data to ensure that when you extract data from your data source, you get rid of the advertisements and you try to get the major information in there, because it’s very valuable in your text analysis.

Of course, the last thing is the timeliness of the data. We all know that people who do stock research get real-time quotes. They get it for a reason, because the newer the quotes are, the surer they can look into the crystal ball and make predictions about the future in a few seconds.

The world is really changing around us. Companies need to look into the crystal ball in the nearer and nearer future. If you are predicting what happens in two years, that doesn’t really matter. You need to know what’s happening tomorrow.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download a copy. Learn more. Sponsor: Kapow Technologies.

November 4th, 2009

HP takes converged infrastructure a notch higher with new data warehouse appliance

Posted by Dana Gardner @ 10:45 am

Categories: Agile Development, BI, Cisco, Cloud computing, Government, HP, Hardware Infrastructure, IBM, IT Management, IT Service Management, Microsoft, Oracle, SOA, SOA Governance, SOA architect, Silicon Valley, Software Development, Software Infrastructure, System Z, VMware, Virtualization, business intelligence, convergence, database, datacenters, governance, mainframe

Tags: Data Warehouse, Hewlett-Packard Co., Data Centers, Storage, Roi/Tco, Databases, Hardware, Data Management, Finance, Managerial Accounting

Hewlett-Packard (HP) on Wednesday announced new products, solutions and services that leaves the technology packaging to them, so users don’t have to.

HP Neoview Advantage, HP Converged Infrastructure Architecture, and HP Converged Infrastructure Consulting Services are designed to help organizations drive business and technology innovations at lower total cost via lower total hassle. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

HP’s measured focus

HP isn’t just betting on a market whim. Recent market research it supported reveals that more than 90 percent of senior business decision makers believe business cycles will continue to be unpredictable for the next few years — and 80 percent recognize they need to be far more flexible in how they leverage technology for business.

The same old IT song and dance doesn’t seem to be what these businesses are seeking. Nearly 85 percent of those surveyed cited innovation as critical to success, and 71 percent said they would sanction more technology investments — if they could see how those investments met their organization’s time-to-market and business opportunity needs.

Cost nowadays is about a lot more than the rack and license. The fuller picture of labor, customization, integration, shared services suppport, data-use-tweaking and inevitable unforeseen gotchas need to be better managed in unison — if that desired agility can also be afforded (and sanctioned by the bean-counters).

HP said its new offerings deliver three key advantages:

• Improved competitiveness and risk mitigation through business data management, information governance, and business analytics
• Faster time to revenue for new goods and services
• The ability to return to peak form, after being compressed or stretched.

The Neoview advantage

First up, HP Neoview Advantage, the new release of the HP Neoview enterprise data warehouse platform, which aims to help organizations respond to business events more quickly by supporting real-time insight and decision-making.

HP calls the performance, capacity, footprint and manageability improvements dramatic and says the software also reduces the total cost of ownership (TCO) associated with industry-standard components and pre-built, pre-tested configurations optimized for warehousing.

HP Neoview Advantage and last year’s Exadata product (produced in partnership with Oracle) seem to be aimed at different segments. Currently, HP Neoview Advantage is a “very high end database,” whereas Exadata is designed for “medium to large enterprises,” and does not scale to the Neoview level, said Deb Nelson, senior vice president, Marketing, HP Enterprise Business.

A converged infrastructure

Next up, HP Converged Infrastructure architecture. As HP describes it, the architecture adjusts to meet changing business needs, specifically what HP calls “IT sprawl,” which it points to as the key culprit in raising technology costs for maintenance that could otherwise be used for innovation.

HP touts key benefits of this new architecture. First, the ability to deploy application environments on the fly through shared service management, followed closely by lower network costs and less complexity. The new architecture is optimized through virtual resource pools and also improves energy integration and effectiveness across the data center by tapping into data center smart grid technology.

Finally, HP is offering Converged Infrastructure Consulting Services that aim to help customers transition from isolated product-centric technologies to a more flexible converged infrastructure. The new services leverage HP’s experience in shared services, cloud computing, and data center transformation projects to let customers design, test and implement scalable infrastructures.

Overall, typical savings of 30 percent in total costs can be achieved by implementing Data Center Smart Grid technologies and solutions, said HP.

With these moves to converged infrastructure, HP is filling out where others are newly treading. Cisco and EMC this week announced packaging partnerships that seek to deliver simiar convergence benefits to the market.

“It’s about experience, not an experiment,” said Nelson.

BriefingsDirect contributor Jennifer LeClaire provided editorial assistance and research on this post.

November 3rd, 2009

Aster Data architects application logic with data for speeded-up analytics processing en masse

Posted by Dana Gardner @ 1:30 pm

Categories: BI, Cloud computing, Developer Tools, Government, Microsoft, Oracle, SOA Governance, Software Development, Software Infrastructure, Testing Tools, business intelligence, convergence, database, datacenters, management

Tags: Business Intelligence, Analytics, Aster Data Version 4.0, Financial Planning, Finance, Dana Gardner

In real estate, the mantra is “location, location, location.” The same could be said for the juxtaposition of applications logic and data. With enterprise data growing at an explosive rate, having applications separate from the mountains of data that they rely on has resulted in massive data movement — increasing latency and restricting due analysis.

Aster Data, which provides massively parallel processing (MPP) data management, has tackled the location problem head-on with the announcement this week of Aster Data Version 4.0, (along with Aster nCluster System 4.0), a massively parallel application-data server that allows companies to embed applications inside an MPP data warehouse. This is designed to speed the processing of terabytes to petabytes of data.

The latest offering from the San Carlos, Calif., company fully parallelizes both data and a wide variety of analytics applications in one system. This provides faster analysis for such data-heavy applications as real-time fraud detection, customer behavior modeling, merchandising optimization, affinity marketing, trending and simulations, trading surveillance, and customer calling patterns.

While both data and applications reside in the same system, they are independent of one another, but both execute as “first-class citizens” with their respective data and application management services.

Resource sharing

The Aster Data Application Server is responsible for managing and coordinating activities and resource sharing in the cluster. It also acts as a host for the application processing and data inside the cluster. In its role as data host, it manages incremental scaling, fault tolerance and heterogeneous hardware for application processing.

Aster Data Version 4.0 provides application portability, which allows companies to take their existing Java, C, C++, C#, .NET, Perl and Python applications, MapReduce-enable them and push them down into the data.

The Dynamic Workload Management (WLM) helps support hundreds of concurrent mixed workloads that can span interactive and batch data queries, as well as application execution. Includes granular rule-based prioritization of workloads and dynamic allocation and re-allocation of resources.

Other features include:

• Trickle feeds for granular data loading and interactive queries with millisecond response times
• New online partition splitting capabilities to allow infinite cost-effective scaling
• Dual-stage query optimizer, which ensures peak performance across hundreds to thousands of CPU cores
• Integrations with leading business intelligence (BI) tools and Hadoop.

More companies want to bring more data to bear on more BI problems. While Aster’s benefits and value may be used for high-end and esoteric analytics uses now, I fully expect that there data-intense architectures will be finding more uses. The price, too, is dropping, making the use of such systems more affordable.

Many of the core users of high-end analytics are also moving on architecture-wise. The systems designed five or more years ago will not meet the needs of five or even a few years from now.

What’s really cool about Aster Data’s approach is the analytics apps can be used, and the languages and query semantics most familiar to users can be used with the new systems and architectures.

I suppose we should also expect more of these analytics engines to become available as services, aka cloud services. That would allow joins of more data sets and they the massive analytics applications can open up even more BI cans of worms.

October 25th, 2009

Application transformation case study targets enterprise bottom line with eye-popping ROI

Posted by Dana Gardner @ 10:44 am

Categories: Application Lifecycle Management, Cloud computing, Developer Tools, Enterprise Java, Government, HP, Hardware Infrastructure, Home, IBM, IT Management, IT Service Management, ITIL, Java, Linux, Open Source, Oracle, Podcasts, SOA, SOA Governance, SOA architect, SaaS, Software Development, Software Infrastructure, System Z, VMware, Virtualization, database, datacenters, governance, mainframe, management

Tags: Legacy Application, Transformation, ROI, End-user Productivity, Podcasts, Roi/Tco, Strategy, Internet, Finance, Managerial Accounting

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download a copy. Learn more. Sponsor: Hewlett-Packard.

This podcast is the first in the series of three to examine Application Transformation: Getting to the Bottom Line. Through a case study, we’ll discuss the rationale and likely returns of assessing the true role and character of legacy applications, and then assess the true paybacks from modernization.

The ongoing impact of the reset economy is putting more emphasis on lean IT — of identifying and eliminating waste across the data-center landscape. The top candidates, on several levels, are the silo-architected legacy applications and the aging IT systems that support them.

Using our case study, we’ll also uncover a number of proven strategies on how to innovatively architect legacy applications for transformation and for improved technical, economic, and productivity outcomes. The podcasts coincidentally run in support of HP virtual conferences on the same subjects:

Register here to attend the Asia Pacific event on Nov. 3. Register here to attend the EMEA event on Nov. 4. Register here to attend the Americas event on Nov. 5.

Here to start us off on our series on the how and why of transforming legacy enterprise applications are Paul Evans, worldwide marketing lead on Applications Transformation at HP, and Luc Vogeleer, CTO for Application Modernization Practice in HP Enterprise Services. The discussion is moderated be me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Evans: When the economic situation hit really hard, we definitely saw customers retreat, and basically say, “We don’t know what to do now. Some of us have never been in this position before in a recessionary environment, seeing IT budgets reduce considerably.”

That wasn’t surprising. … It was obvious that people would retrench and then scratch their heads and say, “Now what do we do?”

Now we’re seeing a different dynamic, … something like a two-fold increase in what you might call “customer interest” [in applications transformation]. The number of opportunities we’re seeing as a company has doubled over the last six or nine months.

If you ask any CIO or IT head, “Is application transformation something you want to do,” the answer is, “No, not really.” It’s like tidying your garage at home. You know you should do it, but you don’t really want to do it. You know that you benefit, but you still don’t want to do it.

This has moved from being something that maybe I should do to something that I have to do, because there are two real forces here. One is the force that says, “If I don’t continue to innovate and differentiate, I go out of business, because my competitors are doing that.” If I believe the economy doesn’t allow me to stand still, then I’ve got it wrong. So, I have to continue to move forward.

Secondly, I have to reduce the amount of money I spend on my innovation, but at the same time I need a bigger payback. I’ve got to reduce the cost of IT. Now, with 80 percent of my budget being dedicated to maintenance, that doesn’t move my business forward. So, the strategic goal is, I want to flip the ratio.

… Today, we’ll hear about a case study — with the Italian Ministry of Instruction, University and Research (MIUR). This customer received an ROI in 18 months. In 18 months, the savings they had made — and this runs into millions of dollars — had been paid for. Their new system, in under 18 months, paid for itself. After that, it was pure money to the bottom-line.

… Our job is to minimize that risk by exposing them to customers who have done it before. They can view those best-case scenarios and understand what to do and what not to do.

Vogeleer: We take a very holistic approach and look at the entire portfolio of applications from a customer. Then, from that application portfolio — depending on the usage of the application, the business criticality of the application, as well as the frequency of changes that this application requires — we deploy different strategies for each application.

We not only focus on one approach of completely re-writing or re-platforming the application or replacing the application with a package, but we go for a combination of all those elements. By doing a complete portfolio assessment, as a first step into the customer legacy application landscape, we’re able to bring out a complete road map to conduct this transformation.

We first execute applications that bring a quick ROI. We first execute quick wins and the ROI and the benefits from those quick wins are immediately reinvested for continuing the transformation. So, transformation is not just one project. It’s not just one shot. It’s a continuous program over time, where all the legacy applications are progressively migrated into a more agile and cost-effective platform.

The Italian Ministry of Instruction, University and Research (MIUR), is the customer we’re going to cover with this case, is a large governmental organization and their overall budget is €55 billion.

This Italian public education sector serves 8 million students from 40,000 schools, and the schools are located across the country in more than 10,000 locations, with each of those locations connected to the information system provided by the ministry.

Very large employer

The ministry is, in fact, one of the largest employers in the world, with over one million employees. Its system manages both permanent and temporary employees, like teachers and substitutes, and the administrative employees. It also supports the ministry users, about 7,000 or 8,000 school employees. It’s a very large employer with a large number of users connected across the country.

Why do they need to modernize their environment? In fact, their system was written in the early 1980s on IBM mainframe architecture. In early 2000, there was a substantial change in Italian legislation, which was called so-called a Devolution Law. The Devolution Law was about more decentralization of their process to school level and also to move the administration processes from the central ministry level into the regions, and there are 20 different regions in Italy.

This change implied a completely different process workflow within their information systems. To fulfill the changes, the legacy approach was very time-consuming and inappropriate. A number of strong application have been developed incrementally to fulfill those new organizational requirements, but very quickly this became completely unmanageable and inflexible. The aging legacy systems were expected to be changed quickly.

In addition to the element of agility to change application to meet the new legislation requirement, the cost in that context went completely out of control. So, the simple, most important objective of the modernization was to design and implement a new architecture that could reduce cost and provide a more flexible and agile infrastructure.

The first step we took was to develop a modernization road map that took into account the organizational change requirements, using our service offering, which is the application portfolio assessment.

From the standard engagement that we can offer to a customer, we did an analysis of the complete set of applications and associated data assets from multiple perspectives. We looked at it from a financial perspective, a business perspective, functionality and the technical perspective.

From those different dimensions, we could make the right decision on each application. The application portfolio assessment ensured that the client’s business context and strategic drivers were understood, before commencing a modernization strategy for a given application in the portfolio.

A business case was developed for modernizing each application, an approach that was personalized for each group of applications and was appropriate to the current situation.

… This assessment phase took about three months with the seven people. From there, we did a first transformation pilot, with a small staff of people in three months.

After the pilot, we went into the complete transform and user-acceptance test, and after an additional year, 90 percent of the transformation was completed. In the transformation, we had about 3,500 batch processes. We had the transformation. We had re-architecting of 7,500 programs. And, all the screens were also transformed. But, that was a larger effort with a team of about 50 people over one year.

… We tried to use automated conversion, especially for non-critical programs, where they’re not frequently changed. That represented 60 percent of the code. This code could be then immediately transferred by removing only the barriers in the code that prevented it from compiling.

All barriers removed

We had also frequently updated programs, where all barriers were removed and code was completely cleaned in the conversion. Then, in critical programs, especially, the conversion effort was bigger than the rewrite effort. Thirty percent of the programs were completely rewritten.

The applications are now accessed through a more efficient web-based user interface, which replaces the green screen and provides improved navigation and better overall system performance, including improved user productivity.

End-user productivity is doubled in terms of the daily operation of some business processes. Also, the overall application portfolio has been greatly simplified by this approach. The number of function points that we’re managing has decreased by 33 percent.

From a financial perspective, there are also very significant results. Hardware and software license and maintenance cost savings were about €400,000 in the first year, €2 million in the second year, and are projected to be €3.4 million this year. This represents a savings of 36 percent of the overall project.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download a copy. Learn more. Sponsor: Hewlett-Packard.

October 21st, 2009

Global study: Hybrid model rules as cloud heats up, SaaS adoption blazing

Posted by Dana Gardner @ 7:46 am

Categories: Akamai, Amazon, Cloud computing, Google, HP, IT Management, IT Service Management, ITIL, Microsoft, Oracle, SAP, SOA, SOA Governance, SOA architect, SaaS, Software Development, Software Infrastructure, VMware, Virtualization, business intelligence, convergence, database, datacenters, governance, mainframe, management

Tags: Software, Software-as-a-service, Avenade, Software As A Service (SaaS), Managed Hosting, Cloud Computing, Tools & Techniques, Emerging Technologies, Management, Dana Gardner

“Cloud” is the game and “hybrid” is the name. A recent global study has encouraging news for cloud-computing enthusiasts, revealing a sharp uptick in the adoption, as well as consideration, of cloud computing. The same study also indicates that those who are adopting cloud aren’t going whole hog, but are taking a hybrid approach — mixing external and internal clouds.

The study, commissioned by global IT consultancy Avanade, showed a surprising increase in the interest in cloud computing, even from a similar study conducted in January of this year. In January, 54 percent of respondents said they had no plans to adopt cloud computing. By September, that percentage had shrunk to 37 percent.

At the same time, the percentage of companies planning or testing cloud computing increased three-fold, going from 3 percent of respondents to 10 percent.

What’s significant in the report is that less than 5 percent of companies are using an all-cloud model. The rest are relying on a hybrid approach, and report security concerns as the chief factor for being cautious.

Nine months ago, 61 percent of respondents indicated that they were using only internal IT systems and today, that number has dropped to 41 percent. At the same time, those using a combined approach on a global level have increased to 54 percent from 33 percent nine months earlier.

The report says it not clear whether the hybrid model will lead to a pure-play adoption at some point.

SaaS is taking off

One aspect of cloud computing that’s finding wide adoption is software as a service (SaaS), with more than half of the respondents worldwide — and 68 percent in the US — reporting that they have adopted SaaS at some level. Despite extremely high satisfaction — more than 90 percent — reliability is still an issue. About 30 percent of respondents said they had lost more than a day of business due to a service outage.

Still, the reliability concerns haven’t dampened users’ enthusiasm for SaaS, and 62 percent of respondents reported that they had plans to move into more SaaS within the next year. However, similar to their experience with cloud, users tend to deliver SaaS applications internally, rather than from the third-party provider.

On a global basis, those who deliver SaaS application internally outnumber those who used a third party by a ratio of 2 to 1. In the US, that increases to 4 to 1. Also, those who do use SaaS often rely on multiple providers, with one third using three or more providers. This leads the report to conclude that there is opportunity in the SaaS market.

Other conclusion from the report:

• Cloud will continue to make significant inroads for the next year, although there won’t be a migration to a full cloud environment.
• The gap is closing between companies with plans to adopt and those without. Avenade sees those curves intersecting in 2011 or 2012.
• Despite the widespread adoption of cloud, there will be some applications that should remain on-premises.
• SaaS adoption will continue to spread and is spreading faster than other technologies have in the past.

The study was conducted by Kelton Research and surveyed 500 C-level and IT executives worldwide.

BriefingsDirect contributor Carlton Vogt provided editorial assistance and research on this post.

October 15th, 2009

Oracle's Fusion Apps finally come out from behind the OpenWorld curtain

Posted by Dana Gardner @ 9:13 am

Categories: Developer Tools, Enterprise Java, HP, Hardware Infrastructure, IBM, Linux, Open Source, Oracle, SOA, SOA Governance, SOA architect, Silicon Valley, Software Development, Software Infrastructure, Sybase, VMware, Virtualization, business intelligence, database, governance, management

Tags: Oracle Corp., Tony, Fusion Apps, Ellison, Service-Oriented Architecture (SOA), Social Networking, Web Services, Middleware, Enterprise Software, Software

This guest post comes courtesy of Tony Baer’s OnStrategies blog. Tony is a senior analyst at Ovum.

By Tony Baer

Like almost every attendee at just-concluded Oracle OpenWorld, the suspense on when Oracle would finally lift the wraps on Fusion Apps was palpable. Staying cool with minimizing our carbon footprint, we weren’t physically at Moscone, but instead watching the webcasts and monitoring the Twitter stream from our home office.

The level of anticipation over Fusion apps was palpable. But it was hardly suspense as it seemed that a good cross-section of Twitterati were either analysts, reference customers, consultants or other business partners who have had their NDA sneak peaks (we had ours back in June), but had to keep our lips sealed until last night.

There was also plenty of impatience for Oracle to finally get on with a message that was being drowned out by its sudden obsession with hardware. Ellison spent most of his keynote time pumping up its Exadata cache memory database storage appliance and issuing a $10 million challenge to IBM that it can’t match Oracle’s database benchmarks on Sun.

Yup, if the Sun acquisition goes trough, Oracle’s no longer strictly a software company, and although the Twiterati counted its share of big iron groupies, the predominant mood was that hardware was a distraction.

“This conference has been hardware heavy from the start. Odd for a software conference,” tweeted Forrester analyst Paul Hamerman. “90 minutes into the keynote, nothing yet on Fusion apps.”

“Larry clearly stalling with all this compression mumbo jumbo,” “Larry please hurry up and tell the world about Fusion Apps, fed up of saying YES it does exist to your skeptics,” and so on read the Twitter stream.

There was fear that Oracle would simply tease us in a manner akin to Jon Stewart’s we’ll have to leave it there dig at CNN: “I am afraid that Larry soon will tell that as time has run out he will tell about Fusion applications in next OOW.” A 20-minute rousing speech from Calif. Gov. Arnold Schwarzenegger served as a welcome relief from Ellison’s newly found affection for big iron toys.

Ellison came back after the Governator pleaded with the audience to stick around awhile and drop some change around California as the state is broke. The break gave him the chance to drift over to Oracle Enterprise Manager, which at least got the conversation off hardware.

Ellison described some evolutionary enhancements where Oracle can track your configurations trough Enterprise Manager and automatically manage patching. As we’ve noted previously, Oracle has compelling solutions for all-Oracle environments, among them being a declarative framework for developing apps and specifying what to monitor and auto-patch.

The main topic

But the spiel on Enterprise Manager provided a useful back door to the main topic, as Ellison showed how it could automate management of the next generation of Oracle apps. Ellison got the audience’s attention with the words, “We are code complete for all of this.”

Well almost everything. Oracle has completed work on all modules except manufacturing.

Ellison then gave a demo that was quite similar to one that we saw under NDA back in the summer. While ERP emerged with and was designed for client/server architectures, Fusion has emerged with a full Java EE and SOA architecture; it is built around Oracle Fusion middleware 11g and uses Oracle BPEL Process Manager to run processes as orchestrations of processes exposed from the Fusion Apps or other legacy applications. That makes the architecture of Fusion Apps clean and flexible.

But at this point, Oracle is not being any more specific about rollout other than to say it would happen sometime next year.

It uses SOA to loosely couple, rather than tightly integrate with other Fusion processes or processes exposed by existing back end applications, which should make Fusion apps more pliant and less prone to outage.

That allows workflows in Fusion to be dynamic and flexible. If an order in the supply chain is held up, the process can be dynamically changed without bringing down order fulfillment processes for orders that are working correctly. It also allows Oracle to embed business intelligence throughout the suite, so that you don’t have to leave the application to perform analytics.

For instance, in an HR process used for locating the right person for a job, you can dig up an employee’s salary history, and instead switching to a separate dashboard, you can instead retrieve and display relevant pieces of information necessary to see comparisons and make a decision.

Fusion’s SOA architecture also allows Oracle to abstract security and access control by relying on its separate, Fusion middleware-based Identity Manager product. The same goes with communications, where instant messaging systems can be pulled in (we didn’t see any integration with Wikis or other Web 2.0 social computing mechanisms, but we assume that they can be integrated as services.). It also applies to user interfaces, where you can use different rich internet clients by taking advantage of Oracle’s ADF framework in JDeveloper.

Oracle concedes the obvious: Outside of the mid-market, there is no greenfield market for ERP, and therefore, Fusion Apps are intended to supplement what you already have, not necessarily replace it.

That includes Oracle’s existing applications, for which it currently promises at least a decade of more support. But at this point, Oracle is not being any more specific about rollouts other than to say it would happen “sometime next year.”

This guest post comes courtesy of Tony Baer’s OnStrategies blog. Tony is a senior analyst at Ovum.

October 5th, 2009

HP roadmap dramatically reduces energy consumption across data centers

Posted by Dana Gardner @ 6:25 pm

Categories: Application Lifecycle Management, Cloud computing, HP, Hardware Infrastructure, IT Management, IT Service Management, ITIL, Podcasts, SOA Governance, Software Development, Software Infrastructure, VMware, Virtualization, database, datacenters, management

Tags: Data Center, Hewlett-Packard Co., Server, Data Centers, Storage, Hardware, Data Management, Dana Gardner

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: Hewlett-Packard.

Gain more insights into data center transformation best practices by downloading free whitepapers at http://www.hp.com/go/dctpodcastwhitepapers.

Producing meaningful, long-term energy savings in IT operations depends on a strategic planning and execution process.

The goal is to seek out long-term gains from prudent, short-term investments, whenever possible. It makes little sense to invest piecemeal in areas that offer poor returns, when a careful cost-benefit analysis for each specific enterprise can identify the true wellsprings of IT energy conservation.

The latest BriefingsDirect podcast discussion therefore targets significantly reducing energy consumption across data centers strategically. In it we examine four major areas that result in the most energy policy bang for the buck — virtualization, application modernization, data-center infrastructure best practices, and properly planning and building out new data-center facilities.

By focusing on these major areas, but with a strict appreciation of the current and preceding IT patterns and specific requirements for each data center, real energy savings — and productivity gains — are in the offing.

To help learn more about significantly reducing energy consumption across data centers, we welcome two experts from HP: John Bennett, worldwide director, Data Center Transformation Solutions , and Ian Jagger, worldwide marketing manager for Data Center Services. The discussion is moderated by me, BriefingsDirect’s Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Bennett: We, as an industry, are full of advice around best practices for what people should be taking a look at. We provide these wonderful lists of things that they should pay attention to — things like hot and cold aisles, running your data center hotter, and modernizing your infrastructure, consolidating it, virtualizing it, and things of that ilk.

The mistakes that customers do make is that they have this laundry list and, without any further insight into what will matter the most to them, they start implementing these things.

The real opportunity is to take a step back and assess the return from any one of these individual best practices. Which one should I do first and why? What’s the technology case and what’s the business case for them? That’s an area that people seem to really struggle with.

… We know very well that modern infrastructure, modern servers, modern storage, and modern networking items are much more energy efficient than their predecessors from even two or three years ago.

… If we look at the total energy picture and the infrastructure itself — in particular, the server and storage environment — one of the fundamental objectives for virtualization is to dramatically increase the utilization of the assets you have.

With x86 servers, we see utilization rates typically in the 10 percent range. So, while there are a lot interesting benefits that come from virtualization from an energy efficiency point of view, we’re basically eliminating the need for a lot of server units by making much better use of a smaller number of units.

So, consolidation and modernization, which reduces the number of units you have, and then multiplying that with virtualization, can result in significant decreases in server and storage-unit counts, which goes a long way toward affecting energy consumption from an infrastructure point of view.

That can be augmented, by the way, by doing application modernization, so you can eliminate legacy systems and infrastructure and move some of those services to a shared infrastructure as well.

We’re talking about collapsing infrastructure requirements by factors of 5, 6, or 10. You’re going from 10 or 20 old servers to perhaps a couple of servers running much more efficiently. And, with modernization at play, you can actually increase that multiplication.

These are very significant from a server point of view on the storage side. You’re eliminating the need for sparsely used dedicated storage and moving to a shared, or virtualized storage environment, with the same kind of cost saving ratios at play here. So, it’s a profound impact in the infrastructure environment.

Jagger: Going back to the original point that John made, we have had the tendency in the past to look at cooling or energy efficiency coming from the technology side of the business and the industry. More recently, thankfully, we are tending to look at that in a more converged view between IT technology, the facility itself, and the interplay between the two.

… Each customer has a different situation from the next, depending on how the infrastructure is laid out, the age of the data center, and even the climatic location of the data center. All of these have enormous impact on the customer’s individual situation.

… If we’re looking, for example, at the situation where a customer needs a new data center, then it makes sense for that customer to look at all the cases put together — application modernization, virtualization, and also data center design itself.

Here is where it all stands to converge from an energy perspective. Data centers are expensive things to build, without doubt. Everyone recognizes that and everybody looks at ways not to build a new data center. But, the point is that a data center is there to run applications that drive business value for the company itself.

What we don’t do a good job of is understanding those applications in the application catalog and the relative importance of each in terms of priority and availability. What we tend to do is treat them all with the same level of availability. That is just inherent in terms of how the industry has grown up in the last 20-30 years or so. Availability is king. Well, energy has challenged that kingship if you like, and so it is open to question.

. . . Converging the facility design with application modernization, takes millions and millions of dollars of data center construction costs, and of course the ongoing operating costs derived from burning energy to cool it at the end of the day.

Now, you could look at designing a facility, where you have within the facility specific PODs (groups of compute resources) that would be designed according to the application catalog’s availability and priority requirements, tone down the tooling infrastructure that is responsible for those particular areas, and just retain specific PODs for those that do require the highest levels of availability.

Just by doing that, by converging the facility design with application modernization, takes millions and millions of dollars of data center construction costs, and of course the ongoing operating costs derived from burning energy to cool it at the end of the day.

… One of the smartest things you can actually do as a business, as an IT manager, is to actually go and talk to your utility company and ask them what rebates are available for energy savings. They typically will offer you ways of addressing how you can improve your energy efficiency within the data center.

That is a great starting point, where your energy becomes measurable. Taking an action on reducing your energy, not just hits your operating cost, but actually allows you to get rebates from your energy company at the same time. It’s a no-brainer.

Bennett: What we are advising customers to do is take a more complete view of the resources and assets that go into delivering business services to the company.

It’s not just the applications and the portfolio. … It’s the data center facilities themselves and how they are optimized for this purpose — both from a data center perspective and from the facility-as-a-building perspective.

In considering them comprehensively in working with the facilities team, as well as the IT teams, you can actually deliver a lot of incremental value — and a lot of significant savings to the organization.

… For customers who are very explicitly concerned about energy and how to reduce their energy cost and energy consumption, we have an Energy Analysis Assessment service. It’s a great way to get started to determine which of the best practices will have the highest impact on you personally, and to allow you to do the cherry-picking.

For customers who are looking at things a little more comprehensively, energy analysis and energy efficiency are two aspects of a data-center transformation process. We have a data center transformation workshop.

Jagger: The premise here is to understand possible savings or the possible efficiency available to you through forensic analysis and modeling. That has got to be the starting point, and then understanding the costs of building that efficiency.

Then, you need a plan that shows those costs and savings and the priorities in terms of structure and infrastructure, have that work in a converged way with IT, and of course the payback on the investment that’s required to build it in the first place.

Gain more insights into data center transformation best practices by downloading free whitepapers at http://www.hp.com/go/dctpodcastwhitepapers.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: Hewlett-Packard.

October 5th, 2009

Web data services extend data access and distribution beyond the RDB-BI straightjacket

Posted by Dana Gardner @ 2:12 pm

Categories: Amazon, BI, Cloud computing, Enterprise 2.0, Google, Internet, Oracle, Podcasts, Software Development, Software Infrastructure, Web Services, Web Technology, business intelligence, database

Tags: Data Service, Web, Data Access, Web Data Service, Pricing, Business Intelligence, Tools & Techniques, Databases, Enterprise Software, Marketing

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Sponsor: Kapow Technologies.

As enterprises seek to gain better insights into their markets, processes, and business development opportunities, they face a daunting challenge — how to identify, gather, cleanse, and manage all of the relevant data and content being generated across the Web.

As the recession forces the need to identify and evaluate new revenue sources, businesses need to capture such web data services for business intelligence (BI) to work better and fuller. In Part 1 of our web data series we discussed how external data has grown in both volume and importance across internal Internet, social networks, portals, and applications in recent years.

Enterprises need to know what’s going on and what’s being said about their markets across those markets. They need to share those web data service inferences quickly and easily across their internal users. The more relevant and useful content that enters into BI tools, the more powerful the BI outcomes — especially as we look outside the enterprise for fast shifting trends and business opportunities.

In this podcast, Part 2 of the series with Kapow Technologies, we identify how BI and web data services come together, and explore such additional subjects as text analytics and cloud computing. So, how to get started and how to affordably manage web data services with BI and business consumers as intelligence and insights?

To find out, we brought together Jim Kobielus, senior analyst at Forrester Research, and Stefan Andreasen, co-founder and chief technology officer at Kapow Technologies. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Kobielus: The more relevant content you bring into your analytic environment the better, in terms of having a single view or access in a unified fashion to all the information that might be relevant to any possible decision you might make. But, clearly, there are lots of caveats, “gotchas,” and trade-offs there.

One of these is that it becomes very expensive to discover, to capture, and to do all the relevant transformation, cleansing, storage, and delivery of all of that content. It becomes very expensive, especially as you bring more unstructured information from your content management system (CMS) or various applications from desktops and from social networks.

… Filtering the fire hose of this content is where this topic of web data services for BI comes in. Web data services describes that end-to-end analytic information pipe-lining process. It’s really a fire hose that you filter at various points, so that the end users turn on their tap and they’re not blown away by a massive stream. Rather, it’s a stream of liquid intelligence that is palatable and consumable.

Andreasen: There is a fire hose of data out there, but some of that data is flowing easily, but some of it might only be dripping and some might be inaccessible.

Think about it this way. The relevant data for your BI applications is located in various places. One is in your internal business applications. Another is your software-as-a-service (SaaS) business application, like Salesforce, etc. Others are at your business partners, your retailers, or your suppliers. Another one is at government. The last one is on the World Wide Web in those tens of millions of applications and data sources.

Accessible via browser

Today, all of this data that I just described is more or less accessible in a web browser. Web data services allow you to access all these data sources, using the interface that the web browser is already using. It delivers that result in a real-time, relative, and relevant way into SQL databases, directly into BI tools, or to even service enabled and encapsulated data. It delivers the benefits that IT can now better serve the analysts need for new data, which is almost always the case.

What’s even more important is that incremental daily improvement of existing reports. Analysts sit there, they find some new data source, and they say, “It would be really good, if I could add this column of data to my report, maybe replace this data, or if I could get this amount of data in real-time rather than just once a week.” So it’s those kinds of improvements that web data services also really can help with.

Kobielus: At Forrester, we see traditional BI as a basic analytics environment, with ad-hoc query, OLAP, and the like. That’s traditional BI — it’s the core of pretty much every enterprise’s environment.

Advanced analytics — building on that initial investment and getting to this notion of an incremental add-on environment — is really where a lot of established BI users are going. Advanced analytics means building on those core reporting, querying, and those other features with such tools as data mining and text analytics, but also complex event processing (CEP) with a front-end interactive visualization layer that often enables mashups of their own views by the end users.

… We see a strong push in the industry toward smashing those silos and bringing them all together. A big driver of that trend is that users, the enterprises, are demanding unified access to market intelligence and customer intelligence that’s bubbling up from this massive Web 2.0 infrastructure, social networks, blogs, Twitter and the like.

Andreasen: Traditionally, for BI, we’ve been trying to gather all the data into one unified, centralized repository, and accessing the data from there. But, the world is getting more diverse and the data is spread in more and different silos. What companies realize today is that we need to get service-level access to the data, where they reside, rather than trying to assemble them all.

…Web data services can encapsulate or wrap the data silos that were residing with their business partners into services — SOAP services, REST services, etc. — and thereby get automated access to the data directly into the BI tool.

… So, tomorrow’s data stores for BI, and today’s as well, is really a combination of accessing data in your central data repositories and then accessing them where they reside. … Think about it. I’m an analyst and I work with the data. I feel I own the data. I type the data in. Then, when I need it in my report, I cannot get it there. It’s like owning the house, but not having the key to the house. So, breaking down this barrier and giving them the key to the house, or actually giving IT a way to deliver the key to the house, is critical for the agility of BI going forward.

Tools are lacking

Today, the IT department often lacks tools to deliver those custom feeds that the line of business is asking for. But, with web data services, you can actually deliver these feeds. The data that IT is asking for is almost always data they already know, see, and work with in the business applications, with the business partners, etc. They work with the data. They see them in the browsers, but they cannot get the custom feeds. With the web data services product, IT can deliver those custom feeds in a very short time.

Kobielus: The user feels frustration, because they go on the Web and into Google and can see the whole universe of information that’s out there. So, for a mashup vision to be reality, organizations have got to go the next step.

… It’s good to have these pre-configured connections through extract, transform and load (ETL) and the like into their data warehouse from various sources. But, there should also be ideally feeds in from various data aggregators. There are many commercial data aggregators out there who can provide discovery of a much broader range of data types — financial, regulatory, and what not.

Also, within this ideal environment there should be user-driven source discovery through search, through pub-sub, and a variety of means. If all these source-discovery capabilities are provided in a unified environment with common tooling and interfaces, and are all feeding information and allowing users to dynamically update the information sets available to them in real-time, then that’s the nirvana.

Andreasen: This is where Kapow and web data services come in, as a disruptive new way of solving a problem of delivering the data — the real-time relevant data that the analyst needs.

The way it works is that, when you work with the data in a browser, you see it visually, you click on it, and you navigate tables and so on. The way our product works is that it allows you to instruct our system how to interact with a web application, just the same way as the line of business user.

…The beauty with web data services is that it’s really accessing the data through the application front end, using credentials and encryptions that are already in place and approved. You’re using the existing security mechanism to access the data, rather than opening up new security holes, with all the risk that that includes.

… This means that you access and work with the data in the world in which the end users see the data. It’s all with no coding. It’s all visual, all point and click. Any IT person can, with our product, turn data that you see in a browser into a real feed, a custom feed, virtually in minutes or in a few hours for something that would typically take days, weeks, or months — or may even be impossible.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Sponsor: Kapow Technologies.