November 18th, 2009

IBM feels cozy on sidelines as Oracle-Sun deal languishes in anti-trust purgatory

Posted by Dana Gardner @ 3:44 am

Categories: Apache, BI, Cloud computing, Eclipse, Enterprise Java, HP, Hardware Infrastructure, IBM, Intellectual Property, Java, Microsoft, Open Source, Oracle, SAP, SOA, Silicon Valley, Software Development, Software Infrastructure, Sybase, System Z, Wall Street, business intelligence, database, datacenters, mainframe

Tags: Oracle Corp., Antitrust, Sun Microsystems Inc., Steve Mills, IBM Corp., Programming Languages, Open Source, Java, Databases, Software Development

You have to know when to hold them, and when to fold them. That’s the not just slightly smug assessment by IBM executives as they reflect — with twinkles in their eyes — on the months-stalled Oracle acquisition of Sun Microsystems, a deal that IBM initially sought but then declined earlier this year.

Chatting over drinks at the end of day one of the Software Analyst Connect 2009 conference in Stamford, Conn., IBM Senior Vice President and IBM Software Group Executive Steve Mills told me last night he thinks the Oracle-Sun deal will go through, but it won’t necessarily be worth $9.50 a share to Oracle when it does.

“He (Oracle Chairman Larry Ellison) didn’t understand the hardware business. It’s a very different business from software,” said Mills.

Mills seemed very much at ease with IBM’s late-date jilt of Sun (Sun was apparently playing hard to get in order to get more than $9.40/share from Big Blue’s coffers). IBM’s stock price these days is homing in on $130, quite a nice turn of events given the global economy.

Sun is trading at $8.70, a significant discount to Oracle’s $9.50 bid, reflecting investor worries about the fate of the deal now under scrutiny by European regulators, Mill’s views notwithstanding.

IBM Software Group Vice President of Emerging Technology Rod Smith noted the irony — perhaps ancient Greek tragedy-caliber irony — that a low market share open source product is holding up the biggest commercial transaction of Sun’s history. “That open source stuff is tricky on who actually makes money and how much,” Smith chorused.

Should Mills’s prediction that Oracle successfully maintains its bid for Sun prove incorrect, it could mean bankruptcy for Sun. And that may mean many of Sun’s considerable intellectual property assets would go at fire-sale prices to … perhaps a few piecemeal bidders, including IBM. Smith just smiled, easily shrugging off the chill (socks in tact) from the towering “IBM” logo ice sculpture a few steps away.

And wouldn’t this hold up go away if Sun and/or Oracle jettisoned MySQL? Is it pride or hubris that makes a deal sour for one mere grape? Was the deal (and $7.4 billion) all about MySQL? Hardly.

Many observers think that Sun’s Java technology — and not its MySQL open source database franchise — should be of primary concern to European (and U.S.) anti-trust mandarins. I have to agree. But Mills isn’t too concerned with Oracle’s probable iron-grip on Java …, err licensing. IBM has a long-term license on the technology, the renewal of which is many years out. “We have plenty of time,” said Mills.

Yes, plenty of time to make Apache Harmony a Java doppelganger — not to mention the Java market-soothing effects of OSGi and Eclipse RCP. [Hey, IBM invented Java for the server for Sun, it can re-invent it for something else ... SAP?]

Unlike some software titans, Mills is clearly not living in a “reality distortion field” when it comes to Oracle’s situation.

“We’re in this for the long haul,” said Mills, noting that he and IBM have have been competing with Oracle since August 1993 when IBM launched its distributed DB2 product. “All of our market share comes at the expense of Oracle’s,” said Mills. “And we love to do benchmarks again Oracle.”

Even as the Fates seem to be on IBM’s side nowadays, the stakes remain high for the users of these high-end database technologies and products. It’s my contention that we’re only now entering the true data-driven decade. And all that data needs to run somewhere. And it’s not going to be in MySQL, no matter who ends up owning it.

November 9th, 2009

Here's why text-based content access and management play crucial roles in real-time BI

Posted by Dana Gardner @ 2:27 pm

Categories: BI, Enterprise 2.0, Google, Government, HP, IBM, Podcasts, Software Development, Software Infrastructure, Web Services, business intelligence, content delivery network, database, management, search

Tags: Dana Gardner

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download a copy. Learn more. Sponsor: Kapow Technologies.

Text-based content and information from across the Web are growing in importance to businesses. The need to analyze web-based text in real-time is rising to where structured data was in importance just several years ago.

Indeed, for businesses looking to do even more commerce and community building across the Web, text access and analytics forms a new mother lode of valuable insights to mine.

As the recession forces the need to identify and evaluate new revenue sources, businesses need to capture such web data services for their business intelligence (BI) to work better, deeper, and faster.

In this podcast discussion, Part 3 of a series on web data services for BI, we discuss how an ecology of providers and a variety of content and data types come together in several use-case scenarios.

In Part 1 of our series we discussed how external data has grown in both volume and importance across the Internet, social networks, portals, and applications. In Part 2, we dug even deeper into how to make the most of web data services for BI, along with the need to share those web data services inferences quickly and easily.

Our panel now looks specifically at how near real-time text analytics fills out a framework of web data services that can form a whole greater than the sum of the parts, and this brings about a whole new generation of BI benefits and payoffs.

To help explain the benefits of text analytics and their context in web data services, we’re joined by Seth Grimes, principal consultant at Alta Plana Corp., and Stefan Andreasen, co-founder and chief technology officer at Kapow Technologies. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Grimes: “Noise free” is an interesting and difficult concept when you’re dealing with text, because text is just a form of human communication. Whether it’s written materials, or spoken materials that have been transcribed into text, human communications are incredibly chaotic … and they are full of “noise.” So really getting to something that’s noise-free is very ambitious.

… It’s become an imperative to try to deal with the great volume of text — the fire hose, as you said — of information that’s coming out. And, it’s coming out in many, many different languages, not just in English, but in other languages. It’s coming out 24 hours a day, 7 days a week — not only when your business analysts are working during your business day. People are posting stuff on the web at all hours. They are sending email at all hours.

If you want to keep up, if you want to do what business analysts have been referring to as a 360-degree analysis of information, you’ve got to have automated technologies to do it.

… There are hundreds of millions of people worldwide who are on the Internet, using email, and so on. There are probably even more people who are using cell phones, text messaging, and other forms of communication.

If you want to keep up, if you want to do what business analysts have been referring to as a 360-degree analysis of information, you’ve got to have automated technologies to do it. You simply can’t cope with the flood of information without them.

Fortunately, the software is now up to the job in the text analytics world. It’s up to the job of making sense of the huge flood of information from all kinds of diverse sources, high volume, 24 hours a day. We’re in a good place nowadays to try to make something of it with these technologies.

Andreasen: … There is also a huge amount of what I call “deep web,” very valuable information that you have to get to in some other way. That’s where we come in and allow you to build robots that can go to the deep web and extract information.

… Eliminating noise is getting rid of all this stuff around the article that is really irrelevant, so you get better results.

The other thing around noise-free is the structure. … The key here is to get noise-free data and to get full data. It’s not only to go to the deep web, but also get access to the data in a noise-free way, and in at least a semi-structured way, so that you can do better text analysis, because text analysis is extremely dependent on the quality of data.

Grimes: … [There are] many different use-cases for text analytics. This is not only on the Web, but within the enterprise as well, and crossing the boundary between the Web and the inside of the enterprise.

Those use-cases can be the early warning of a Swine flu epidemic or other medical issues. You can be sure that there is text analytics going on with Twitter and other instant messaging streams and forums to try to detect what’s going on.

… You also have brand and reputation management. If someone has started posting something very negative about your company or your products, then you want to detect that really quickly. You want early warning, so that you can react to it really quickly.

We have some great challenges out there, but . . . we have great technologies to respond to those challenges.

We have a great use case in the intelligence world. That’s one of the earliest adopters of text analytics technology. The idea is that if you are going to do something to prevent a terrorist attack, you need to detect and respond to the signals that are out there, that something is pending really quickly, and you have to have a high degree of certainty that you’re looking at the right thing and that you’re going to react appropriately.

… Text analytics actually predate BI. The basic approaches to analyzing textual sources were defined in the late ’50s. Actually, there is a paper from an IBM researcher from 1958, that defines BI as the analysis of textual sources.

…[Now] we want to take a subset of all of the information that’s out there in the so-called digital universe and bring in only what’s relevant to our business problems at hand. Having the infrastructure in place to do that is a very important aspect here.

Once we have that information in hand, we want to analyze it. We want to do what’s called information extraction, entity extraction. We want to identify the names of people, geographical location, companies, products, and so on. We want to look for pattern-based entities like dates, telephone numbers, addresses. And, we want to be able to extract that information from the textual sources.

Suitable technologies

All of this sounds very scientific and perhaps abstruse — and it is. But, the good message here is one that I have said already. There are now very good technologies that are suitable for use by business analysts, by people who aren’t wearing those white lab coats and all of that kind of stuff. The technologies that are available now focus on usability by people who have business problems to solve and who are not going to spend the time learning the complexities of the algorithms that underlie them.

Andreasen: … Any BI or any text analysis is no better than the data source behind it. There are four extremely important parameters for the data sources. One is that you have the right data sources.

There are so many examples of people making these kind of BI applications, text analytics applications, while settling for second-tier data sources, because they are the only ones they have. This is one area where Kapow Technologies comes in. We help you get exactly the right data sources you want.

The other thing that’s very important is that you have a full picture of the data. So, if you have data sources that are relevant from all kinds of verticals, all kinds of media, and so on, you really have to be sure you have a full coverage of data sources. Getting a full coverage of data sources is another thing that we help with.

Noise-free data

We already talked about the importance of noise-free data to ensure that when you extract data from your data source, you get rid of the advertisements and you try to get the major information in there, because it’s very valuable in your text analysis.

Of course, the last thing is the timeliness of the data. We all know that people who do stock research get real-time quotes. They get it for a reason, because the newer the quotes are, the surer they can look into the crystal ball and make predictions about the future in a few seconds.

The world is really changing around us. Companies need to look into the crystal ball in the nearer and nearer future. If you are predicting what happens in two years, that doesn’t really matter. You need to know what’s happening tomorrow.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download a copy. Learn more. Sponsor: Kapow Technologies.

November 4th, 2009

HP takes converged infrastructure a notch higher with new data warehouse appliance

Posted by Dana Gardner @ 10:45 am

Categories: Agile Development, BI, Cisco, Cloud computing, Government, HP, Hardware Infrastructure, IBM, IT Management, IT Service Management, Microsoft, Oracle, SOA, SOA Governance, SOA architect, Silicon Valley, Software Development, Software Infrastructure, System Z, VMware, Virtualization, business intelligence, convergence, database, datacenters, governance, mainframe

Tags: Data Warehouse, Hewlett-Packard Co., Data Centers, Storage, Roi/Tco, Databases, Hardware, Data Management, Finance, Managerial Accounting

Hewlett-Packard (HP) on Wednesday announced new products, solutions and services that leaves the technology packaging to them, so users don’t have to.

HP Neoview Advantage, HP Converged Infrastructure Architecture, and HP Converged Infrastructure Consulting Services are designed to help organizations drive business and technology innovations at lower total cost via lower total hassle. [Disclosure: HP is a sponsor of BriefingsDirect podcasts.]

HP’s measured focus

HP isn’t just betting on a market whim. Recent market research it supported reveals that more than 90 percent of senior business decision makers believe business cycles will continue to be unpredictable for the next few years — and 80 percent recognize they need to be far more flexible in how they leverage technology for business.

The same old IT song and dance doesn’t seem to be what these businesses are seeking. Nearly 85 percent of those surveyed cited innovation as critical to success, and 71 percent said they would sanction more technology investments — if they could see how those investments met their organization’s time-to-market and business opportunity needs.

Cost nowadays is about a lot more than the rack and license. The fuller picture of labor, customization, integration, shared services suppport, data-use-tweaking and inevitable unforeseen gotchas need to be better managed in unison — if that desired agility can also be afforded (and sanctioned by the bean-counters).

HP said its new offerings deliver three key advantages:

• Improved competitiveness and risk mitigation through business data management, information governance, and business analytics
• Faster time to revenue for new goods and services
• The ability to return to peak form, after being compressed or stretched.

The Neoview advantage

First up, HP Neoview Advantage, the new release of the HP Neoview enterprise data warehouse platform, which aims to help organizations respond to business events more quickly by supporting real-time insight and decision-making.

HP calls the performance, capacity, footprint and manageability improvements dramatic and says the software also reduces the total cost of ownership (TCO) associated with industry-standard components and pre-built, pre-tested configurations optimized for warehousing.

HP Neoview Advantage and last year’s Exadata product (produced in partnership with Oracle) seem to be aimed at different segments. Currently, HP Neoview Advantage is a “very high end database,” whereas Exadata is designed for “medium to large enterprises,” and does not scale to the Neoview level, said Deb Nelson, senior vice president, Marketing, HP Enterprise Business.

A converged infrastructure

Next up, HP Converged Infrastructure architecture. As HP describes it, the architecture adjusts to meet changing business needs, specifically what HP calls “IT sprawl,” which it points to as the key culprit in raising technology costs for maintenance that could otherwise be used for innovation.

HP touts key benefits of this new architecture. First, the ability to deploy application environments on the fly through shared service management, followed closely by lower network costs and less complexity. The new architecture is optimized through virtual resource pools and also improves energy integration and effectiveness across the data center by tapping into data center smart grid technology.

Finally, HP is offering Converged Infrastructure Consulting Services that aim to help customers transition from isolated product-centric technologies to a more flexible converged infrastructure. The new services leverage HP’s experience in shared services, cloud computing, and data center transformation projects to let customers design, test and implement scalable infrastructures.

Overall, typical savings of 30 percent in total costs can be achieved by implementing Data Center Smart Grid technologies and solutions, said HP.

With these moves to converged infrastructure, HP is filling out where others are newly treading. Cisco and EMC this week announced packaging partnerships that seek to deliver simiar convergence benefits to the market.

“It’s about experience, not an experiment,” said Nelson.

BriefingsDirect contributor Jennifer LeClaire provided editorial assistance and research on this post.

November 3rd, 2009

Aster Data architects application logic with data for speeded-up analytics processing en masse

Posted by Dana Gardner @ 1:30 pm

Categories: BI, Cloud computing, Developer Tools, Government, Microsoft, Oracle, SOA Governance, Software Development, Software Infrastructure, Testing Tools, business intelligence, convergence, database, datacenters, management

Tags: Business Intelligence, Analytics, Aster Data Version 4.0, Financial Planning, Finance, Dana Gardner

In real estate, the mantra is “location, location, location.” The same could be said for the juxtaposition of applications logic and data. With enterprise data growing at an explosive rate, having applications separate from the mountains of data that they rely on has resulted in massive data movement — increasing latency and restricting due analysis.

Aster Data, which provides massively parallel processing (MPP) data management, has tackled the location problem head-on with the announcement this week of Aster Data Version 4.0, (along with Aster nCluster System 4.0), a massively parallel application-data server that allows companies to embed applications inside an MPP data warehouse. This is designed to speed the processing of terabytes to petabytes of data.

The latest offering from the San Carlos, Calif., company fully parallelizes both data and a wide variety of analytics applications in one system. This provides faster analysis for such data-heavy applications as real-time fraud detection, customer behavior modeling, merchandising optimization, affinity marketing, trending and simulations, trading surveillance, and customer calling patterns.

While both data and applications reside in the same system, they are independent of one another, but both execute as “first-class citizens” with their respective data and application management services.

Resource sharing

The Aster Data Application Server is responsible for managing and coordinating activities and resource sharing in the cluster. It also acts as a host for the application processing and data inside the cluster. In its role as data host, it manages incremental scaling, fault tolerance and heterogeneous hardware for application processing.

Aster Data Version 4.0 provides application portability, which allows companies to take their existing Java, C, C++, C#, .NET, Perl and Python applications, MapReduce-enable them and push them down into the data.

The Dynamic Workload Management (WLM) helps support hundreds of concurrent mixed workloads that can span interactive and batch data queries, as well as application execution. Includes granular rule-based prioritization of workloads and dynamic allocation and re-allocation of resources.

Other features include:

• Trickle feeds for granular data loading and interactive queries with millisecond response times
• New online partition splitting capabilities to allow infinite cost-effective scaling
• Dual-stage query optimizer, which ensures peak performance across hundreds to thousands of CPU cores
• Integrations with leading business intelligence (BI) tools and Hadoop.

More companies want to bring more data to bear on more BI problems. While Aster’s benefits and value may be used for high-end and esoteric analytics uses now, I fully expect that there data-intense architectures will be finding more uses. The price, too, is dropping, making the use of such systems more affordable.

Many of the core users of high-end analytics are also moving on architecture-wise. The systems designed five or more years ago will not meet the needs of five or even a few years from now.

What’s really cool about Aster Data’s approach is the analytics apps can be used, and the languages and query semantics most familiar to users can be used with the new systems and architectures.

I suppose we should also expect more of these analytics engines to become available as services, aka cloud services. That would allow joins of more data sets and they the massive analytics applications can open up even more BI cans of worms.

October 16th, 2009

What's on your watch list? Forrester identifies 15 key technologies for enterprise architects

Posted by Dana Gardner @ 8:29 am

Categories: Agile Development, Amazon, Apple, Application Lifecycle Management, BI, Cisco, Cloud computing, Enterprise 2.0, Enterprise Java, Google, HP, IT Management, IT Service Management, ITIL, Microsoft, Open Source, Oracle, SOA, SOA Governance, SOA architect, SaaS, Security, Software Development, Software Infrastructure, VMware, Virtualization, Web Services, Web Technology, business intelligence, convergence, datacenters, governance, iPhone, management

Tags: Forrester Research Inc., Operational Planning, Pricing, Business Intelligence, Tools & Techniques, Strategy, Business Operations, Marketing, Enterprise Software, Software

Riding the right — or wrong — technology wave can help — or really, really hurt — your business. Moving at the right time can be the critical factor between the two outcomes.

Yet new technologies come down the pike at alarming speed. Deciding which will fizzle and which will sizzle — and when — can be a daunting and ongoing task. What’s an enterprise architect to do?

Forrester Research has tried to sort things out with a new report, “The Top 15 Technology Trends EA Should Watch.” And, if even limiting the selection to 15 sounds like a lot to keep your eye on, Forrester has grouped them into five major “themes,” and has ranked the technologies by their impact, newness and complexity.

Calling “impact” the most important criterion, the report says this considers whether the technology will deliver new business capabilities or allow IT to improve business performance.

“Newness” comes in second because it’s likely that enterprises will have to gear up to learn new processes and the processes themselves are prone to rapid evolution. “Complexity” places other demands on the business, requiring more time to learn operations that are more complex than others.

The five themes identified by Forrester, along with their associated technologies, are:

• Social computing in and around the enterprise
  • Collaboration platforms become people-centric
  • Customer community platforms integrate with business apps
  • Telepresence gains widespread use
• Process-centric data and intelligence
  • Business intelligence goes real time
  • Master data management matures
  • Data quality services become real-time
• Restructured IT services platforms
  • SaaS will be ubiquitous for packaged apps
  • Cloud-based platforms that become standard infrastructure and platform as a service
  • Client virtualization is ubiquitous
• Agile and fit-to-purpose applications
  • Business rules processing moves to the mainstream
  • BPM will be Web 2.0-enabled
  • Policy-based SOA becomes predominant
  • Security will be data- and content-based
• Mobile as the new desktop
  • Apps and business processes go mobile
  • Mobile networks and devices gain more power

The technologies range from real-time business intelligence (BI) with a very high impact, high newness and high complexity to data- and content-based security, which scored a medium in all three categories. I guess that keep my friend Jim Koblielus busy for some time.

Forrester limited the report to a three-year horizon for two reasons. First, it represents the planning horizon for most firms and, second, any technology that won’t have an effect in less than three years may be interesting, but it’s not actionable.

The report also says that we’re entering a new phase of technology innovation. This analysis is based on Forrester’s finding that technology change goes through two waves. The first involves innovation and growth. This features a rapid evolution of the technology and rapid uptake by businesses. The second phase is refinement and redesign, in which technologies are only incrementally improved.

I hear a lot these day about “inflection points” in the IT market. I hear folks point to the hockey stick growth effect coming for netbooks/thin clients/desktop virtualization/Windows 7. I like to add the smartphones and Android-o-hones to that category too.

And even if the cloud is a slow burn, rather than hockey stick, the importance of business processes supported by services supported by all the old and new suspects is huge. I call the ability to refine and adapt business processes as the big productivity maker of the next decade — supported by IT as services.

Perhaps the new Moore’s Law is less about systems, and more about what people do with the services those systems enable. What do you think?

Incidentally, the full report is available for download from Forrester.

BriefingsDirect contributor Carlton Vogt provided editorial assistance and research on this post.

October 7th, 2009

Survey says slow, kludgy business processes hamper competitiveness

Posted by Dana Gardner @ 11:07 am

Categories: BI, Enterprise 2.0, Microsoft, Open Source, Progress Software, SOA, SOA Governance, SOA architect, Software Development, Software Infrastructure, Web Technology, business intelligence, datacenters, governance

Tags: Business Process, Progress Software Corp., Survey, Operational Planning, Podcasts, Tools & Techniques, Business Operations, Internet, Management, Dana Gardner

Corporations, are your business processes slowing you down? If so, you are in good company. Seventy-two percent of organizations say their business processes take too long and need to be streamlined.

So says a new independent survey conducted by Vanson Bourne for Progress Software.

The survey had a single goal, to determine the tools and processes large companies have put in place to support operational responsiveness and the ability to make “real-time” decisions. Vanson Bourne surveyed 400 large companies in the United States and Western Europe to develop its findings.

The bottom line: An overwhelming majority of businesses still feel they have a ways to go before they are equipped to respond to market or customer changes quickly enough to compete well in a global marketplace.

“The quest for faster operational responsiveness is becoming more urgent now that external factors such as social networking have boosted speed of response,” says Dr. Giles Nelson, senior director of strategy at the Apama division of Progress Software. “If organizations can’t keep up with the pace of customer feedback, they will find themselves exposed to competitive threats.”

I recently reached a similar conclusion in a podcast discussion with IT analyst Howard Dresner, with an emphasis on business intelligence (BI) in the stew of real-time requirements. Other firms I’ve worked with, such as Active Endpoints and BP Logix, call the value “nimble” or the ability to quick orchestrate and adapt processes.

[UPDATE: TIBCO today delivered its iProcess Spotfire product for real-time BI aligned to business process management.]

Sure is a lot of emphasis on real-time data, analysis and process reactivity nowadays! No process like the present, I always say. [Disclosure: TIBCO and Progress are sponsors of BriefingsDirect podcasts.]

On average, 22 percent of U.S. companies surveyed by Vanson Bourne admitted that, by the time they noticed it, they had missed the opportunity to react competitively to a change or trend affecting one of their processes. A lack of information seems to be fueling the problem. More than half of companies identified information gaps in decision-making as a cause.

The good news is that surveyed companies have solutions to the information gap in mind, namely access to real-time data. Ninety-four percent of companies cited the importance of real-time data – and the majority of those companies are making moves to gather it. Some 82 percent are planning to invest in real-time technology by mid-2010 in an effort to speed up internal processes, they said.

As Nelson at Apama sees it, bad news now travels very quickly – and companies need to make sure they’re not stuck in the slow lane when it comes to responding to customer issues.

“The overwhelming majority of people we spoke to recognize the importance of responding quickly to customers and to be much more responsive to changes in market conditions. Unfortunately, in most cases at present the process and information reporting infrastructure can’t match that vision,” Nelson says. “Business Event Processing is becoming the way of dealing with this decision-making lag.”

I’d add a bit more. What we’re actually seeing is that corporations now see that they must be able to analyze and act in Internet time. Many of us webby and social-media types have known that for some time, but the urgency has now hit the mainstream bricks (not just the clicks).

Furthermore, the payoffs from becoming a real-time-oriented organization will go far beyond knowing what’s being said about you on Twitter. As the economy has shown in the last year, those who can move fast and move well will survive and thrive. The others will find themselves in a downward spiral.

BriefingsDirect contributor Jennifer LeClaire provided editorial assistance and research on this post.

October 5th, 2009

Web data services extend data access and distribution beyond the RDB-BI straightjacket

Posted by Dana Gardner @ 2:12 pm

Categories: Amazon, BI, Cloud computing, Enterprise 2.0, Google, Internet, Oracle, Podcasts, Software Development, Software Infrastructure, Web Services, Web Technology, business intelligence, database

Tags: Data Service, Web, Data Access, Web Data Service, Pricing, Business Intelligence, Tools & Techniques, Databases, Enterprise Software, Marketing

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Sponsor: Kapow Technologies.

As enterprises seek to gain better insights into their markets, processes, and business development opportunities, they face a daunting challenge — how to identify, gather, cleanse, and manage all of the relevant data and content being generated across the Web.

As the recession forces the need to identify and evaluate new revenue sources, businesses need to capture such web data services for business intelligence (BI) to work better and fuller. In Part 1 of our web data series we discussed how external data has grown in both volume and importance across internal Internet, social networks, portals, and applications in recent years.

Enterprises need to know what’s going on and what’s being said about their markets across those markets. They need to share those web data service inferences quickly and easily across their internal users. The more relevant and useful content that enters into BI tools, the more powerful the BI outcomes — especially as we look outside the enterprise for fast shifting trends and business opportunities.

In this podcast, Part 2 of the series with Kapow Technologies, we identify how BI and web data services come together, and explore such additional subjects as text analytics and cloud computing. So, how to get started and how to affordably manage web data services with BI and business consumers as intelligence and insights?

To find out, we brought together Jim Kobielus, senior analyst at Forrester Research, and Stefan Andreasen, co-founder and chief technology officer at Kapow Technologies. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Kobielus: The more relevant content you bring into your analytic environment the better, in terms of having a single view or access in a unified fashion to all the information that might be relevant to any possible decision you might make. But, clearly, there are lots of caveats, “gotchas,” and trade-offs there.

One of these is that it becomes very expensive to discover, to capture, and to do all the relevant transformation, cleansing, storage, and delivery of all of that content. It becomes very expensive, especially as you bring more unstructured information from your content management system (CMS) or various applications from desktops and from social networks.

… Filtering the fire hose of this content is where this topic of web data services for BI comes in. Web data services describes that end-to-end analytic information pipe-lining process. It’s really a fire hose that you filter at various points, so that the end users turn on their tap and they’re not blown away by a massive stream. Rather, it’s a stream of liquid intelligence that is palatable and consumable.

Andreasen: There is a fire hose of data out there, but some of that data is flowing easily, but some of it might only be dripping and some might be inaccessible.

Think about it this way. The relevant data for your BI applications is located in various places. One is in your internal business applications. Another is your software-as-a-service (SaaS) business application, like Salesforce, etc. Others are at your business partners, your retailers, or your suppliers. Another one is at government. The last one is on the World Wide Web in those tens of millions of applications and data sources.

Accessible via browser

Today, all of this data that I just described is more or less accessible in a web browser. Web data services allow you to access all these data sources, using the interface that the web browser is already using. It delivers that result in a real-time, relative, and relevant way into SQL databases, directly into BI tools, or to even service enabled and encapsulated data. It delivers the benefits that IT can now better serve the analysts need for new data, which is almost always the case.

What’s even more important is that incremental daily improvement of existing reports. Analysts sit there, they find some new data source, and they say, “It would be really good, if I could add this column of data to my report, maybe replace this data, or if I could get this amount of data in real-time rather than just once a week.” So it’s those kinds of improvements that web data services also really can help with.

Kobielus: At Forrester, we see traditional BI as a basic analytics environment, with ad-hoc query, OLAP, and the like. That’s traditional BI — it’s the core of pretty much every enterprise’s environment.

Advanced analytics — building on that initial investment and getting to this notion of an incremental add-on environment — is really where a lot of established BI users are going. Advanced analytics means building on those core reporting, querying, and those other features with such tools as data mining and text analytics, but also complex event processing (CEP) with a front-end interactive visualization layer that often enables mashups of their own views by the end users.

… We see a strong push in the industry toward smashing those silos and bringing them all together. A big driver of that trend is that users, the enterprises, are demanding unified access to market intelligence and customer intelligence that’s bubbling up from this massive Web 2.0 infrastructure, social networks, blogs, Twitter and the like.

Andreasen: Traditionally, for BI, we’ve been trying to gather all the data into one unified, centralized repository, and accessing the data from there. But, the world is getting more diverse and the data is spread in more and different silos. What companies realize today is that we need to get service-level access to the data, where they reside, rather than trying to assemble them all.

…Web data services can encapsulate or wrap the data silos that were residing with their business partners into services — SOAP services, REST services, etc. — and thereby get automated access to the data directly into the BI tool.

… So, tomorrow’s data stores for BI, and today’s as well, is really a combination of accessing data in your central data repositories and then accessing them where they reside. … Think about it. I’m an analyst and I work with the data. I feel I own the data. I type the data in. Then, when I need it in my report, I cannot get it there. It’s like owning the house, but not having the key to the house. So, breaking down this barrier and giving them the key to the house, or actually giving IT a way to deliver the key to the house, is critical for the agility of BI going forward.

Tools are lacking

Today, the IT department often lacks tools to deliver those custom feeds that the line of business is asking for. But, with web data services, you can actually deliver these feeds. The data that IT is asking for is almost always data they already know, see, and work with in the business applications, with the business partners, etc. They work with the data. They see them in the browsers, but they cannot get the custom feeds. With the web data services product, IT can deliver those custom feeds in a very short time.

Kobielus: The user feels frustration, because they go on the Web and into Google and can see the whole universe of information that’s out there. So, for a mashup vision to be reality, organizations have got to go the next step.

… It’s good to have these pre-configured connections through extract, transform and load (ETL) and the like into their data warehouse from various sources. But, there should also be ideally feeds in from various data aggregators. There are many commercial data aggregators out there who can provide discovery of a much broader range of data types — financial, regulatory, and what not.

Also, within this ideal environment there should be user-driven source discovery through search, through pub-sub, and a variety of means. If all these source-discovery capabilities are provided in a unified environment with common tooling and interfaces, and are all feeding information and allowing users to dynamically update the information sets available to them in real-time, then that’s the nirvana.

Andreasen: This is where Kapow and web data services come in, as a disruptive new way of solving a problem of delivering the data — the real-time relevant data that the analyst needs.

The way it works is that, when you work with the data in a browser, you see it visually, you click on it, and you navigate tables and so on. The way our product works is that it allows you to instruct our system how to interact with a web application, just the same way as the line of business user.

…The beauty with web data services is that it’s really accessing the data through the application front end, using credentials and encryptions that are already in place and approved. You’re using the existing security mechanism to access the data, rather than opening up new security holes, with all the risk that that includes.

… This means that you access and work with the data in the world in which the end users see the data. It’s all with no coding. It’s all visual, all point and click. Any IT person can, with our product, turn data that you see in a browser into a real feed, a custom feed, virtually in minutes or in a few hours for something that would typically take days, weeks, or months — or may even be impossible.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Sponsor: Kapow Technologies.

October 1st, 2009

Kapow and StrikeIron team-up to offer web data services capabilities to SMBs

Posted by Dana Gardner @ 6:40 am

Categories: Amazon, BI, Cloud computing, Developer Tools, Enterprise 2.0, Google, IT Management, IT Service Management, Internet, Microsoft, SOA, SOA Governance, SOA architect, SaaS, Software Development, Software Infrastructure, Web Services, Web Technology, business intelligence, datacenters, management

Tags: Data Service, StrikeIron, Web, Team, Small And Medium Business, Kapow Technologies, Kapow, Channel Management, Smb/Sme, RSS

Kapow Technologies has joined forced with StrikeIron to give small and medium-sized businesses (SMB) a leg up in accessing, using, and sharing Web-based data.

Kapow’s Web Data Services 7.0.0 will allow SMBs to wrap any Web site or Web application into RSS feeds or REST Web services. [Disclosure: Kapow is a sponsor of BriefingsDirect podcasts.]

Under Kapow’s strategic partnership with StrikeIron, Web Data Services 7.0.0, which is available immediately, will be offered on StrikeIron’s Web Services Catalog. The software-as-a-service (SaaS) distribution engine allows developers and business users to integrate live data from private and public Web applications and Web sites.

By using Kapow’s latest offering, SMBs that need enterprise-class Web data services access and quality will have automated and structured access without resorting, as they did previously, to cutting and pasting the data from a Web browser. [Learn more about Web data services and business inteligence.]

Kapow’s “no coding” technology enables companies to rapidly build, test and deploy standard RSS data feeds and REST web services delivery of real-time web data directly into common business applications such as Microsoft Excel, NetSuite or Salesforce as well as any RSS feed reader.

Kapow can also deliver feeds and services directly to any application builder that can access data in standard RSS, JSON and XML format, including IBM Mashup Center, IBM Rational EGL, JackBe and WaveMaker.

The feeds and services are constructed by a visual point-and-click desktop tool that enables users to create “robots” that automate the navigation and interaction with any Web application or Web site, providing secure and reliable access to the underlying data and business logic. This enables the collection of web intelligence and market data in real-time.

Under the terms of the agreement, Kapow will maintain full technical and operational responsibility for Kapow Web Data Services, including enhancements and upgrades. StrikeIron will provide the commercialization capabilities, handling all customer relationship management functions, including sales, billing, and account support.

September 21st, 2009

Web data services extend business intelligence depth and breadth across social, mobile, web domains

Posted by Dana Gardner @ 11:50 am

Categories: BI, Cloud computing, Enterprise 2.0, Google, IBM, Microsoft, Mobile, Oracle, Podcasts, SaaS, Software Development, Software Infrastructure, Web Services, Web Technology, business intelligence, database, management, search

Tags:

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: Kapow Technologies.

See popular event speaker Howard Dresner’s latest book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, or visit his website.

The latest BriefingsDirect podcast discussion on the future of business intelligence (BI) — and on bringing more information from more sources into an analytic process, and thereby getting more actionable intelligence out.

The explosion of information from across the Web, from mobile devices, inside of social networks, and from the extended business processes that organizations are now employing all provide an opportunity, but they also provide a challenge.

This information can play a critical role in allowing organizations to gather and refine analytics into new market strategies, better buying decisions, and to be the first into new business development opportunities. The challenge is in getting at these Web data services and bringing them into play with existing BI tools and traditional data sets.

This is the first in a series of podcasts, looking at the future of BI and how Web data services can be brought to bear on better business outcomes.

So, what are Web data services and how can they be acquired? Furthermore, what is the future of BI when these extended data sources are made into strong components of the forecasts and analytics that enterprises need to survive the recession and also to best exploit the growth that follows?

Here to help us explain the benefits of Web data services and BI is Howard Dresner, president and founder of Dresner Advisory Services, and Ron Yu, vice president of marketing at Kapow Technologies. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Dresner: BI is really about empowering end users, as well as their respective organizations, with insight, the ability to develop perspective. In a downturn, what better time is there to have some understanding of some of the forces that are driving the business?

Of course, it’s always useful to have the benefit of insight and perspective, even in good times. But, it tends to go from being more outward-focused during good times, focused on markets and acquiring customers and so forth, to being more introspective or internally focused during the bad times, understanding efficiencies and how one can be more productive.

So, BI always has merit and in a downturn it’s even more relevant, because we are really less tolerant of being able to make mistakes. We have to execute with even greater precision, and that’s really what BI helps us do.

… The future is about focusing on the information and those insights that can empower the individuals, their respective departments, and the enterprise to stay aligned with the mission of that organization.

… If you’re trying to develop [such] perspective, bringing as much relevant data or information to bear is a valuable thing to do. A lot of organizations focus just on lots of information. I think that you need to focus on the right information to help the organization and individuals carry out the mission of that organization.

There are lots of information sources. When I first started covering this beat 20 years ago, the available information was largely just internal stores, corporate stores, or databases of information. Now, a lot of the information that ought to be used, and in many cases, is being used, is not just internal information, but is external as well.

There are syndicated sources, but also the entire World Wide Web, where we can learn about our customers and our competitors, as well as a whole host of sources that ought to considered, if we want to be effective in pursuing new markets or even serving our existing customers.

Yu: I fully agree with Howard. It’s all about the right data and, given the current global and market conditions, enterprises have cut really deep — from the line of business, but also into the IT organizations. However, they’re still challenged with ways to drive more efficiencies, while also trying to innovate.

The challenges that are being presented are monumental where traditional BI methods and tools are really providing powerful analytical capabilities. At the same time, they’re increasingly constrained by limited access to not only relevant data, but how to get timely access to data.

What we see are pockets of departmental use cases, where marketing departments and product managers are starting to look outside in public data sources to bring in valuable information, so they can find out how the products and services are doing in the market.

… Inclusive BI essentially includes new and external data sources for departmental applications, but that’s only the beginning. Inclusive BI is a completely new mindset. For every application that IT or line of business develops, it just creates another data silo and another information silo. You have another place that information is disconnected from others.

… There is effectively a new class of BI applications as we have been discussing, that depends on a completely different set of data sources. Web data services is about this agile access and delivery of the right data at the right time.

With different business pressures that are surfacing everyday, this leads to a continuous need for more and more data sources.

… Critical decision-making requires, as Howard was saying earlier, that all business information is easily leveraged whenever it’s needed. But today, each application is separate and not joined. This makes the line of business and decision- making very difficult, and it’s not in real time.

An easier way

As this dynamic business environment continues to grow, it’s completely infeasible for IT to update their existing data warehouses or to build a new data mart. That can’t be the solution. There has to be an easier way to access and extract data exactly where it resides, without having to move data back and forth from data bases, data marts, and data warehouses, which effectively becomes snapshot.

… Web data services provides immediate access to the delivery of this critical data into the business user’s BI environment, so that the right timely decisions can be made. It effectively takes these dashboards, reporting, and analytics to the next level for critical decision-making. So when we look deeper into this and how is this actually playing out, it’s all about early and precise predictions.

Dresner: … Some IT organizations have become pretty inflexible. They are focused myopically on some internal sources and are not being responsive to the end user.

To the extent that they can find new tools like Web data services to help them be more effective and more efficient, they are totally open to giving line of business self-service capabilities.

You need to be careful not to suffer from what I call BI myopia, where we are focused just on our internal corporate systems or our financial systems. We need to be responsive. We need to be inclusive of information that can respond to the user’s needs as quickly as possible, and sometimes the competency center is the right approach.

I have instances where the users do wrest control and, in my latest book, I have four very interesting case studies. Some are focused on organizations, where it was more IT driven. In other instances, it was business operations or finance driven.

Yu: … For example, in leading financial services companies, what they’re looking for is on this theme of early and precise predictions. How can you leverage information sources that are publicly available, like weather information, to be able to assess the precipitation and rainfall and even the water levels of lakes that directly contribute to hydroelectricity?

If we can gather all that information, and develop a BI system that can aggregate all this information and provide the analytical capabilities, then you can make very important decisions about trading on energy commodities and investment decisions.

Web data services effectively automates this access and extraction of the data and metadata so that IT doesn’t have to.

Web data services effectively automates this access and extraction of the data and metadata and things of that nature, so that IT doesn’t have to go and build a brand new separate BI system every time line of business comes up with a new business scenario.

… It’s about the preciseness of the data source that the line of business already understands. They want to access it, because they’re working with that data, they’re viewing that data, and they’re seeing it through their own applications every single day.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: Kapow Technologies.

See popular event speaker Howard Dresner’s latest book, Profiles in Performance: Business Intelligence Journeys and the Roadmap for Change, or visit his website.

September 1st, 2009

XDAS standard aims to empower IT audit trails from across complex events, perhaps clouds

Posted by Dana Gardner @ 8:39 am

Categories: .NET, Agile Development, Amazon, BI, Cloud computing, Enterprise Java, Google, Government, HP, IBM, IT Management, IT Service Management, Java, Open Source, Podcasts, SOA, SOA Governance, SOA architect, SaaS, Software Development, Software Infrastructure, Testing Tools, Virtualization, Web Services, business intelligence, datacenters, governance, management

Tags: The Open Group, Audit, Information Technology, XDAS, Audit Trail, Financial Accounting, Finance, Dana Gardner

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: The Open Group.

Welcome to the latest BriefingsDirect podcast discussion, recorded at The Open Group’s 23rd Enterprise Architecture Practitioners Conference and the associated 3rd Security Practitioners Conference in Toronto.

We’re going to take a look at an emerging updated standard called XDAS, which looks at audit trail information from a variety of systems and software across the enterprise IT environment.

This is an emerging standard that’s being orchestrated through The Open Group, but it’s an open-source standard that is hopefully going to help in compliance and regulatory issues and in improving automation of events across heterogeneous environments. This could be increasingly important, as we get deeper into virtualization and cloud computing.

Here to help us drill into XDAS (see a demo now), we’re joined by Ian Dobson, director of the Security Forum for The Open Group, as well as Joël Winteregg, CEO and co-founder of NetGuardians. The discussion is moderated by me, Dana Gardner, principal analyst at Interarbor Solutions.

Here are some excerpts:

Dobson: We actually got involved way back in ’90s, in 1998, when we published the Distributed Audit Service (XDAS) Standard. It was, in many ways, ahead of its time, but it was a distributed audit services standard. Today’s audit and logging requirements are much more demanding than they were then. There is a heightened awareness of everything to do with audit and logging, and we see a need now to update it to meet today’s needs. So that’s why we’ve got involved now.

A key part of this is event reporting. Event reports have all sorts of formats today, but that makes them difficult to consume. Of course, we then generate events so that they can be consumed in useful ways. So, we’re aiming the new audit standard from XDAS to be something that defines an interoperable event-reporting format, so that they can be consumed equally by everybody who needs to know.

The XDAS standard developers are well aware of, and closely involved in, the related Common Event Expression (CEE) standard development activity in Mitre. Mitre’s CEE standard has a broader scope than XDAS, and XDAS will fit very well into the Event Reporting Format part of CEE.

We are therefore also participating in the CEE standard development to achieve this and more, so as to deliver to the audit and logging community an authoritative single open standard that they can adopt with confidence.

Winteregg: My company is working in the area of audit event management. We saw that it was a big issue to collect all these different audit trails from each different IT environment.

We saw that, if it was possible to have a single and standard way to represent all this information, that would be much easier and relevant for IT user and for a security officer to analyze all this information, in order to find out what the exact issues are, and to troubleshoot issue in the infrastructure, and so on. That’s a good basis for understanding what’s going on the whole infrastructure in the company.

There is no uniform way to represent this information, and we thought that this initiative would be really good, because it will bring something uniform and universal that will help all the IT users to understand what is going on.

In distributed environments, it’s really hard to track a transaction, because it starts on a specific component, then it goes through another one, and to a cloud. You don’t know exactly where everything is happening. So, the only way to track these transactions or to track the accountability in such an environment would be through some transaction identifiers, and so on.

For auditors or administrator, it is really costly to understand this information and use it

You will be able to track the who, the what, and the when in the whole IT infrastructure, which is really important these days . . .

in order to get relevant information for management to have metrics and to understand what’s really happening on the IT infrastructure.

Audit information deals a lot with the accountability of the different transactions in an enterprise IT infrastructure. The real logs, which are modulated to develop strong meaning for debugging applications, may be providing the size of buffers or parameters of an application. Audit trails are much more business oriented. That means that you will have a lot of accountability information. You will be able to track the who, the what, and the when in the whole IT infrastructure, which is really important these days with all these different regulations, like Sarbanes-Oxley (SOX) and the others.

With a standard like XDAS, it will be much easier for a company to be in compliance with regulations, because there will be really clear and specific interfaces from all the different vendors to these generated audit trails.

The standard will be open, but there is a Java implementation of that standard called XDAS for J, which is a Java Library. This implementation is open source and business friendly. That means that you can use it in some proprietary software without having to then provide your software as an open-source software. So, it is available for business software too, and all the code is open. You can modify it, look at it, and so on. It’s on the Codehaus platform.

We’re waiting for some feedback from vendors and users about how it is easy to use, how helpful it is, and if there are maybe some use cases — if the scope is too wide, too narrow, etc. We’re open to every comment about the current standard.

Listen to the podcast. Find it on iTunes/iPod and Podcast.com. View a full transcript or download the transcript. Learn more. Sponsor: The Open Group.