November 4th, 2008

Open APIs reach new high water mark as the Web evolves

Posted by Dion Hinchcliffe @ 12:27 pm

Categories: Badges, Business Models, Cloud computing, Cost-effective scalability, Design Patterns, Encouraging Unintended Uses, Enterprise Mashups, Enterprise Web 2.0, Gadgets, Global SOA, JSON, Lightweight Service Models, Mashups, Network Effects, Network effects, Open APIs, REST, Radical Decentralization, Right To Remix, SOA, SOAP, Small Pieces, Loosely Joined, Web 2.0, Web 2.0 Platforms, Web as Platform, Web services, Widgets

Tags: Web, API, Channel Management, Internet, Marketing, Dion Hinchcliffe, Business Leader

Late last week an important milestone for the Internet was quietly reached as the number of available open Web APIs crossed the 1,000 mark, according to the popular API tracking service, Programmable Web.

We are nearing the time when opening our supply chains across the Web isn’t just a good idea, it will be essential for competitive survival. While still seemingly small in number compared to the number of traditional Web sites that exist, open APIs have become an increasingly vital story for Web startups and traditional firms alike to cost effectively partnership, expand the reach of their products (and especially their data), and drive their network effect deeply across the Web.

It’s now almost uncommon to see a new Web product that doesn’t sport a shiny new API so that other online products can integrate the pieces they like into new experiences and offerings. In short, APIs allow a Web application or online business to have thousands of points of presence in other products, instead of just one.

Though APIs were pioneered by many of the original, successful firms on the Web including eBay and Amazon, which can both cite considerable returns for their efforts, it’s only been in the last couple of years that APIs have been taken seriously in a widespread way by the Web community and have become a new competency area.

In my discussions with many companies, one of the biggest obstacles to adopting APIs is a lack of understanding of what a non-visual Web presence looks like and how to build a business model around it. Business leaders are much more likely to understand investment in a traditional Web site, which they are familiar with and understand somewhat, than in an online software development kit, which is more developer-centric and which they are much less likely to fully appreciate, even though APIs can often have more strategic value than a Web site.

The good news is that emerging case studies and the impressive numbers from Amazon earlier this year are showing the the way and there has been a noticeable change in attitude and uptick in interest since cloud computing became such a big topic over the summer.

Yesterday, Programmable Web’s John Musser summarized some of the more interesting findings

Read the rest of this entry »

September 6th, 2008

The WOA story emerges as better outcomes sought for SOA

Posted by Dion Hinchcliffe @ 2:04 pm

Categories: ATOM, Ajax, Business Models, Customer Self-Service, Encouraging Unintended Uses, Enterprise Mashups, Enterprise Web 2.0, Global SOA, Governance, JSON, Lightweight Service Models, Mashups, Network Effects, Network effects, Open APIs, REST, Radical Decentralization, Rich Internet Applications (RIA), Right To Remix, SOA, SOAP, SaaS, Small Pieces, Loosely Joined, Tolerance Continuum, Web 2.0, Web as Platform, Web services, Web-Oriented Architecture (WOA), Widgets

Tags: Web, Business, SOA, Organization, WOA, WOA Story, REST, Service-Oriented Architecture (SOA), Web Services, Middleware

Over the summer the enterprise IT blogosphere was swept up in a conversation around the concepts that many are calling Web-Oriented Architecture, or WOA. A different way to think about service-oriented architecture, WOA extolls a different but related set of technologies, in particular how to apply them in specific ways to connect our systems together into the solutions we need to take on our daily business challenges. WOA offers the exciting and fast-growth promise of the Web 2.0 world, while SOA has been seen as struggling and encountering low engagement in most organizations.

For those just joining the conversation, SOA is the most common set of top-level organizing principles and technologies that enterprises use to organize and connect their IT systems. However, SOA is increasingly in the firing line for less-than-stellar results and lack of business alignment. Few promising solutions for this have emerged lately, with the increasingly notable exception of WOA. WOA describes a compelling new focus that can address many existing SOA issues, but is sometimes at odds with traditional IT and business thinking.

Along with different technology emphasis, WOA offers a compelling new perspective on service uptake and consumption and offers potent ways of thinking about business models that can directly drive innovation and growth. Even better, we can now point to existing WOA success stories, albeit most of them in the online world. In short, SOA (of which WOA is a part) hasn’t looked this interesting in years. But like most new ideas, it inevitably faces challenges from the old guard.

For its own part, far from being a boring, back-office story about plumbing and infrastructure, SOA has actually seen better results than most of the enterprise architecture models that came before it. However, these returns have been fairly lackluster compared to what most business were actually looking for and what SOA practitioners wanted and were actively trying to achieve, certainly when any measurements of the ROI were taken. My detailed WOA overview last April tells the story: The Burton Group ultimately concluded earlier this year that “that SOA is not working in most organizations” based on extensive conversations with clients.

I’ve covered this territory a number of times in the past, most notably with an in-depth exploration of What is WOA?, but the story remains the same: WOA is being driven by the widespread success that lightweight Web services — and particularly their use in open APIs — are having on the open Web. The broad lesson that has been dawning on the enterprise architecture world this year is that this is what’s actually working in terms of what SOA has been trying to accomplish, but with a uniquely different approach.

Explore several WOA success stories and how they are driving SOA.

The tide seems to be turning in terms of the industry’s perspective of WOA as well. Respected SOA expert David Linthicum recently asked “SOA out, WOA in?” and seemed to think it was, noting it will take a long time, like SOA did, to make inroads in the enterprise despite its widespread adoption on the Internet. ZDNet’s own Joe McKendrick recently noted that “WOA wins hands-down over SOA in popularity contest” and Dave Rosenberg recently discussed WOA on CNET and took it as a forgone conclusion. And this is a key point: Many organizations I talk to are already using some WOA to some degree on the ground today, it’s just not being promoted like traditional SOA is, thereby missing the benefit of the support, documentation, guidance, management, and infrastructure/tools support needed to fully flourish.

We have started to see traditional organizations begin to offer WOA-friendly services to the world at large. For example, the World Bank recently opened its Web API to developers using the increasingly popular Mashery service, which allows an organization to outsource their WOA. Of course, WOA can be used solely inside the firewall but some of the most interesting scenarios involve integration with business partners, on demand in a very agile, lightweight fashion.

And in the end, this is the challenge. The use of WOA on the technology side is only interesting if there is support for the business for the scenarios it encourages. You could convert all your Web services from SOAP or REST and be fully ready for the resulting stream of consumer and enterprise mashups, API customers, and hundreds of new business partners, but not if you’ve not redesigned your business a bit. This is also one key reason WOA isn’t synonymous with REST. WOA is architecture, both technical and business, while REST is a style building WOA services. The implications of WOA also go beyond REST to include other Web-oriented scenarios such as widgets, browser-based interfaces, and so on.

WOA entails both technology and business change

Unfortunately, many businesses have not yet absorbed the lessons of the Web 2.0 era and still look at the Web simply as a way to deliver Web pages. This limited view and understanding of the Web’s potential means that most organizations do not have it on the radar to link themselves together in the enterprise-wide and Web-wide ecosystems of creation and integration that WOA can enable. SOA has always been about connecting systems and people together and — at long last — we have a clear path to potentially wonderful outcomes in terms of unintended uses. This includes the ability to access business opportunities inside of time windows which would previously have been unattainable with our traditional, heavyweight SOA models. But only if we truly change the way we think about how to leverage the network.

One last thing, it’s important to remember that no small system can sustain contact with a large system for very long without being fundamentally changed by it. This is what is happening with businesses (the small system, no matter how large) and the Web today (the big system.) The intrinsic nature of the Web is driving major changes in how we create network-based products and services and is inexorably turning us into Web-oriented businesses. Businesses that want to be successful on this network without understanding its fundamental nature and capabilities are only delaying the time it takes to reach the full potential the Web offers.

In this way, WOA often describes network business models (such as open Web APIs) that often seem very foreign to non-Internet businesses but are powerfully aligned with the way that the Web works. These models are almost certainly essential to be successful and flourish in the modern competitive landscape on our networks today. In this way, too many organizations will ignore adding a WOA aspect to their SOA work until it’s too late and the ability to generate strong network effects in their industry is greatly reduced.

WOA is just one of a set of transformative new distribution models for network-based systems.

So how do organizations start down this route to investigate the WOA way of doing SOA and seeing if it works for them?

Like many aspects of Web 2.0, WOA is not complex or overly expensive, it’s a way of thinking about interacting over the network and all the classic SOA principles still apply, which just create and expose them differently.

1. Learn about WOA. Study the technology (HTTP, REST, syndication, open Web APIs, widgets, metadata documentation, Ajax, mashups, JSON, etc.), as well as the business and implementation side, including partner ecosystems, developer support sites, monetization, and chargebacks.
2. Adapt WOA to your organization. Every organization will have a landscape of existing SOA approaches and technologies that WOA approaches will need to be added to. Furthermore, WOA does little good unless you’re willing to use it for what it does well: Provide the fuel for RIA-powered portal applications, enterprise mashups, your public APIs, and so on. Begun working through how WOA security will work in your organization (inline or through HTTPS, for example) and other key starter issues that are (hopefully) already described in your SOA governance documents.
3. Conduct a pilot. Validate the items in #2 with a small pilot. Select a mashup platform that works well for your organization and try it out. WOA enables SOA to be used in a much more agile, open, and effective manner, with the right tools involved but only in an environment that supports it all the way through the “stack” from browser, server, database, development tools, and management infrastructure.

What are your thoughts on WOA? Will this finally be where the rubber meets the road for many SOAs?

July 23rd, 2007

A bumper crop of new mashup platforms

Posted by Dion Hinchcliffe @ 12:49 am

Categories: Ajax, Business Models, Cost-effective scalability, Customer Self-Service, Design Patterns, Encouraging Unintended Uses, Enterprise Mashups, Enterprise Web 2.0, Enterprise Wikis, Gadgets, Global SOA, Governance, JSON, Mashups, Open APIs, Products, RSS, Rich Internet Applications (RIA), Right To Remix, SOA, SaaS, Situational Software, Small Pieces, Loosely Joined, The Long Tail, Two-Way Web, Web 2.0, Web 2.0 Platforms, Web as Platform, Web services, Widgets, Wikis

Tags: Web, Platform, Mashup, Dion Hinchcliffe

While application developers tend to roll their eyes at the concept of end-user mashups, they remain one of the more promising new trends in software development this year. And while it’s certainly true it’s early days yet for mashups, the tools that enable them remaining rather limited, seems to be changing as I regularly come across compelling new mashup platforms as well as upgrades to existing ones that show what will be possible soon. And for now, as evidenced recently in the McKinsey Web 2.0 in business survey where 21% of organizations globally said they are using or planning to use mashups, there appears to be considerable demand for mashups at the enterprise level even though the majority of existing offerings are primarily aimed at the consumer space. Is this disconnect resolving with the current crop of offerings? Let’s take a look.

In today’s mashup world, the apparent business potential of highly accessible and easy-to-use mashup creation tools like Yahoo! Pipes and Microsoft’s PopFly is still undermined by their apparent lack of readiness for the enterprise. Mashups could theoretically allow business users to move — when appropriate — from their current so-called “end-user development tools” such as Microsoft Excel that are highly isolated and poorly integrated to much more deeply integrated models that are more Web-based and hence more open, collaborative, reusable, shareable, and in general make better use of existing sources of content and functionality. Remember, business workers still spend a significant amount of time manually integrating together the data in their ever increasing number of business applications. Tools that could let thousands of workers solve their situational software integration problems on the spot themselves, instead of waiting (sometimes forever) for IT to provide a solution, is indeed a potent vision.

So what’s typically missing from today’s mashup platforms to make them both useful and desirable in the enterprise? While no one knows for sure, since mashups are just starting to be considered seriously in many organizations, it generally boils down to 1) deep access to existing enterprise services and data/content repositories, 2) SaaS-style Web-based mashup assembly and use, 3) assembly models that are truly end-user friendly with very little training required, 4) a credible management and maintenance story for IT departments that must support a flood of public end-user built and integrated apps, and last but certainly not least, 5) mashup products that address important questions about mashups and enterprise security. None of these are particularly easy to solve, which is most likely why mashups haven’t been more prevalent before now.

This latter issue of security — in terms of reliably securing applications that are created largely out of other services and applications — can’t be understated and will likely determine whether an mashup platform can even be considered for adoption in a given organization. This is particularly crucial since the Global SOA, the vast landscape of open functionality and content on the Web, now provides a truly massive yet rather security-challenged set of source of material for enterprise mashups. The question here is whether Web apps that are assembled by users — and not developers or security experts — and that combine capabilities from a wide variety of sources including the open Web can ever be made safe enough for most businesses? That’s an important open question and one that few of the mashup platforms listed below spend much time addressing.

Are mashups really a major new development model? Read a detailed discussion.

The answers to these questions will inevitably shake out as the existing mashups products get applied to real business problems and the industry collectively learns what capabilities and approaches are needed for them to be successful. And I don’t expect it will be a one size-fits-all either; mashups can be approached many different ways, from the pure service mashup models of RSSBus and Kapow’s RoboMaker to the innovative yet very end-user friendly wiki model that IBM’s QEDWiki takes.

I’ve been been tracking many of these new or evolving mashup platforms and thought I’d compile my take of the leading players in the mashup space today, particularly given the number of new or significantly upgraded products in the last few months. To make the cut, all the products listed below had to allow live integration of functionality or content (data) over a network, provide an easy-to-use development model that is theoretically accessible by end-users, be available in at least beta form, and either consume and/or produce Web-based applications and services. Using this refined selection model, you’ll see this list looks a bit different from last year’s round-up of mashup platforms. Yet despite the removal of a few products, the list is bigger than Read the rest of this entry »

January 5th, 2007

2007: The year enterprises open their SOAs to the Internet?

Posted by Dion Hinchcliffe @ 12:24 pm

Categories: ATOM, Business Process Management, Convergence, Enterprise Mashups, Enterprise Web 2.0, Global SOA, Governance, JSON, Lightweight Service Models, Mashups, Orchestration, REST, RSS, Rich Internet Applications (RIA), SOA, SOAP, SaaS, Small Pieces, Loosely Joined, Web 2.0, Web services, Web-Oriented Architecture (WOA)

Tags:

Those that follow the trends on the Internet and the trends within the enterprise have long noticed a very similar direction in both spaces for a while now; a push to move their software to a real services model.  The reasons for this push seem straightforward: easier integration between systems, a better foundation for building new applications, dynamic business process automation that crosses organizational boundaries, and better management and monitoring of IT systems. 

By having a fully-functioning services infrastructure there's even the potential of such esoteric sounding things as BAM, or business activity monitoring, which reasons that if you expose most of your software as services in a standard way, you can finally see what your business is actually doing, while it's doing it.  Thus, services are currently the hot topic as new IT systems come out-of-the-box with open services and older ones are retrofitted with them.  Consequently, as we shall see shortly, 2007 will likely be the tipping point for most organizations to make service-oriented architectures one of their top priorities.

But when it comes to Internet services and enterprise services, the big difference has always been the disparate drivers in these two very different environments:

1) Control - Individual parts of enterprises typically feel unrewarded for opening up and sharing their services and data.  Contrast this with the Web, where opening up and sharing with others is usually rewarded by new customers in terms of users, increased business, advertisers, etc.  Simply put: The culture of most enterprises, with their hierarchical reporting structures, often just does not encourage the freewheeling distribution of information and access to functionality the way that the Web does.

2) Cost structure - In the enterprise, IT systems are typically an overhead cost, or worse, belong to a particular business unit, with either owner driving up overhead costs or unfairly taxing a business unit when a system opens up within the enterprise.  On the Internet however, services via open APIs already have a natural built-in cost recovery mechanism; nominally a fee for usage or other positive motivation such as industry adoption or ecosystem establishment.

3) Mandate vs. necessity - Many enterprises have been trying to make the move to SOAs for several years now, often with SOA projects that attempt to bring consistent standards, technologies, products, and schemas across the organization.  These are typically top-down initiatives (and indeed, almost need to be) that because of their necessary infrastructure focus then all too often fail to connect directly with the operational reality of running a business.  This results in the typical query from the CEO: "How is my SOA specifically helping me run my business better?".  On the Web however, having an open API has nearly become a matter of survival.  If a SaaS product or Web 2.0 site fails to offer an API, it's practically a deal killer since data can't easily be migrated in and out of the service, integration with other existing systems isn't an option, and 3rd party add-ons will not be available.

These drivers give us the state of affairs we seem to be witnessing on the Internet and in the enterprise currently; longer than expected ramp-up and adoption for

Read the rest of this entry »

November 7th, 2006

Web 2.0 Summit: IBM evolves vision of SOA and Web 2.0

Posted by Dion Hinchcliffe @ 6:34 pm

Categories: ATOM, Ajax, Business Models, Business Process Management, Collaboration, Enterprise 2.0, Enterprise Mashups, Enterprise Web 2.0, Global SOA, Governance, JSON, Lightweight Service Models, Mashups, REST, RSS, Rich Internet Applications (RIA), Right To Remix, SOA, SOAP, SaaS, Small Pieces, Loosely Joined, Social Networking, Social Software, Two-Way Web, User Generated Content, WS-*, Web 2.0, Web as Platform, Web services

Tags:

One of the most consistent trends on the Internet is the rise of open APIs and the applications built on top of them, known as mashups.  Programmable Web currently lists over 300 APIs that can be used for everything from building Web sites on top of Google Maps to using Amazon's powerful infrastructure APIs for storage and cluster computing. The underlying trend: The desire to easily remix the vast pool of high value data and services on the Web today into useful new solutions, at home and perhaps even in the enterprise.

In a late morning session today at the Web 2.0 Summit, IBM's Rod Smith painted a compelling picture of this mashup trend combined with the emerging edge of enterprise IT. The result?  A surprisingly close relationship between the somewhat stodgy world of service-oriented architecture (SOA) and the bustling world of consumer Web 2.0.

The convergence of SOA and Web 2.0, two highly interrelated trends that are very focused on 1) connecting people and systems together easily, 2) making software and data available for reuse via services, and 3) building new value upon the foundation of existing information resources and IT assets.  This convergence is something I've been following quite closely for a while, particularly in some key exploratory blogs and articles late last year.

SOA/Web 2.0 convergence is important topic since many of the biggest challenges in enterprise IT are actually being solved today out on the Web, particular around the best way to engage users, deliver highly usable software, integrate systems, and achieve high levels of reuse and adoption.  So it was fascinating to hear the experiences of Rod and his team over the last year, where they've visited over a 100 customers sites and taken the temperature of CIOs in terms of how Web 2.0 can help them with their business problems.

The story is a fascinating one and in this first part of two, we'll take a look at what Rod and his team at IBM is uncovering in the business trenches in terms of

Read the rest of this entry »

September 2nd, 2006

Assembling great software: A round-up of eight mashup tools

Posted by Dion Hinchcliffe @ 12:32 pm

Categories: ATOM, Ajax, Business Process Management, Collaboration, Encouraging Unintended Uses, Enterprise 2.0, Enterprise Mashups, Enterprise Web 2.0, Global SOA, Governance, JSON, Lightweight Service Models, Mashups, Orchestration, Products, REST, RSS, Rich Internet Applications (RIA), Right To Remix, SOA, SOAP, SaaS, Small Pieces, Loosely Joined, Web 2.0, Web as Platform, Web services

Tags:

There is a frequently recurring piece of software development lore that plays on the fact that good programmers are supposed to be lazy.  In these stories, a good programmer will take a frequently recurring, monotonous task (like testing) and instead of doing it by hand, will instead write a piece of code once that will do the task for them, thereby automating it for future use.

Put another way, instead of carrying out the work by hand, a lazy programmer will spend 95% of the time allotted to the work by developing code that will carry it out for them, and the last 5% of the time will be spent running it to get the actual work done.  Then, every time the task must be carried out in the future, software can be directed to complete it swiftly and automatically.

While this is a simplified model (one must ask who checks the work every time to make sure it’s right, how is the code maintained over time, and so on), it’s also one of the significant motivations behind the drive for end-user mashups; applying this very same concept of task automation to daily work and life.

How many routine tasks could we get out of our way if we had powerful task automation tools that almost anyone could use?  How many

Read the rest of this entry »

May 13th, 2006

Using SaaS and Web 2.0 for business automation

Posted by Dion Hinchcliffe @ 3:05 pm

Categories: Business Models, Business Process Management, Collaboration, Collective Intelligence, Customer Self-Service, Enterprise Web 2.0, Global SOA, JSON, Lightweight Service Models, Mashups, Network Effects, Products, REST, Rich Internet Applications (RIA), SOA, SOAP, SaaS, WS-*, Web 2.0, Web as Platform, Web services

Tags:

I’ve been spending a lot of time lately looking at solutions for automated business processes that are based on the online, low-barrier, and highly collaborative worlds of SaaS and Web 2.0.  Primarily, this is part of my exploration of using Web 2.0 in the enterprise, sometimes called Enterprise 2.0, but which we call Enterprise Web 2.0 here.

Read the overview of our exploration of Web 2.0 strategies in the Enterprise, or read the strategies explored so far, which have covered enterprise approaches for blogs and wikis.

This area is of importance because good, effective Business Process Management has been one of the holy grails of enterprise software for years now.  Traditional software development has repeatedly yielded BPM results that are too heavyweight, brittle, hard-to-change, and not responsive to the business.  And like with so many aspects of Web 2.0, looking at the successful models in the highly Darwinian Petri dish of the Web gives us many suggestions on how to do it better:  Dynamic languages that make mashing together functionality both inexpensive and easy, low-impedence and highly scalable integration models such as JSON and REST instead of SOAP or WS-*, peer production techniques that harness the users as the users operate the system, self-service IT and the list goes on.

Harnessing Web 2.0 techniques for business process integration, automation, and management, particularly around highly-repetitive, transactionalprocesses will allow more time for tacit interactions, the high value knowledge work that many workers can’t spend time doing because of the overhead of tedious, low-value transactional work. Tacit interactions are perceived as one of the biggest remaining avenuesfor achieving higher worker productivity.

And if any of this is true, we should be seeing

Read the rest of this entry »

February 14th, 2006

The power of simplicity at the edge

Posted by Dion Hinchcliffe @ 2:36 am

Categories: JSON, Lightweight Service Models, Mashups, Products, REST, RSS, Right To Remix, SOA, SOAP, Tolerance Continuum, Web 2.0, Web services

Tags:

One of the demonstrations I like to give to show the power of simple services is to use Iridesco’s Suprglu to instantly wire together Web services into a new and useful service on the spot.  This is the simplest possible mashup you can create but it does combine useful data from multiple sources into a single view.  You can do it yourself: copy and paste a few feeds from a few sources of interest and you’re off and running. 

Every decision to add unneeded structure or ceremony to the consumption of data in your organization is a tax that reduces the value that data offers.

This visceral demonstration shows the promise of using simple data formats like RSS to let anyone create useful new information sources and services in a straightforward manner.  I say services because Suprglu converts the resulting mashup into yet another RSS feed that can in turn be used and recombined anywhere on the Web.

In fact, service recombination, mashups, and remixing are going to force a lot of us to rethink our existing services to enable a composable Web. So too will the scalability, reliability, feedback loops, and the associated intellectual property issues, of the nascent remix culture.  Rethinking is necessary because so much of our approach in the IT world is still mired in the world of intended uses.  As we all know, most software in the enterprise is designed for specific requirements defined well ahead of time.  While service-oriented architecture (SOA) finally encourages second-order uses of services, I would suggest that this encouragement is just not forceful enough.

Note that Web 2.0 and SOA are primarily concerned about the edges of systems.  This could be the experience edge where the user interacts with with software, typically HTML, Ajax, Flash, etc.  Or it could be the services edge that provides pure information to other systems, such as HTTP, REST, RSS, and SOAP.  What happens inside isn’t revealed, nor should it.  This is the separation of interface from implementation that we’ve tried for so long to achieve.  And since software is increasingly made from pushing these edges together, we should seek to understand the best ways to do that.

Like agile software development discovered,

Read the rest of this entry »