The Semantic Web

September 1st, 2009

Oracle delivers native support for Thomson Reuters' OpenCalais service

Posted by Paul Miller @ 5:15 am

Categories: Commercialisation, Semantic Web, Semantic Web Companies

Tags: Thomson Reuters Corp., Oracle Corp., Paul Miller

Thomson Reuters and Oracle today announced support for the media giant’s OpenCalais metadata generation service within release 2 of Oracle Spatial 11g. The integration gives Oracle users and developers direct access to OpenCalais’ natural language processing (NLP) capabilities.

More importantly, perhaps, direct integration with an Enterprise product such as Oracle’s database says much about how far the semantic technology community has come in being able to offer solutions capable of scaling - robustly - to meet Enterprise-scale demands.

Xavier Lopez, Oracle’s Director for Spatial and Semantic Technologies, is quoted in Thomson Reuters’ press release;

“This interoperability lets users quickly process documents in different formats (such as Microsoft Word and Adobe PDF), to extract semantic metadata that can be used for more semantically complete searches in Oracle11g.”

August 10th, 2009

Moving Data.gov towards the Semantic Web

Posted by Paul Miller @ 3:46 am

Categories: Open Data, Podcasts, Research, Semantic Web, Semantic Web People, Standards, Talking Semantics, Web 3.0

Tags: Government, Semantic Web, Podcasts, RDF, XML, Internet, Software/Web Development, Web Development, Paul Miller

Government transparency in all its forms would appear to be very much in vogue at present, spanning everything from the Obama administration’s Data.gov portal and Prime Ministerial pronouncements in the UK Parliament to municipal proclamations of openness in Vancouver and compelling grass-roots demonstrations by activists and even newspapers.

At the heart of many of today’s initiatives lie programmes to surface Government data for use and re-use by third parties. The ‘open’ in ‘Open Data’ is, of course, a very loaded term, and I’ve looked before at some of the ways in which data might become ‘open’ whilst remaining effectively useless. Nevertheless, Governments’ current enthusiasm for being seen to embrace transparency should certainly be both welcomed and encouraged, and there are real opportunities to work with Government in ensuring that today’s transparency fervour continues undiminished, whether by omission or commission.

Given the complex and varied nature of the data involved, and the obvious linkages between the entities (you and I, our communities, our schools, our hospitals) described in numerous different databases, there’s a clear opportunity for technologies and approaches from the Semantic Web community to play a significant role in simplifying the whole process of moving these legacy databases online.

Already interested in Open Government from previous roles, and (obviously!) committed to encouraging real-world adoption of semantic technologies, I’ve spent some time recently talking to a number of those involved. A number of those conversations are now available as podcasts, and I’ll continue to seek out fresh examples and perspectives to share.

My most recent podcast conversation, released today, is with Professor Jim Hendler and Dr Li Ding of the Tetherless World Constellation at Rensselaer Polytechnic Institute in Troy, NY. The team at Rensselaer have been working with some of the US Federal Government’s data sets on Data.gov, and so far they’ve converted sixteen data sets from their original form, resulting in 2,927,398,352 freely available RDF triples and a number of demonstration applications.

Other conversations already released in the series include;

• David Eaves, talking about Vancouver’s commitment to Open Data
• John Sheridan, Head of e-Services at the UK Government’s Office of Public Sector Information, talking about his Department’s efforts to get Government data online
• Mark Birbeck, talking about work with the UK Government’s Central Office of Information to embed lightweight RDFa into workflows and web pages

Each offers an example of ways in which ‘open data’ contributes to Government transparency, or to increasing the value of the massive sunk investment in collecting, managing and curating the data upon which Governments depend. The Semantic Web’s notion of Linked Data (whether actually in RDF or not! ) offers a means to increase the utility of the data we have, without a massive programme of reengineering the systems used to manage it. The examples we see today, and the work of the individuals and teams with whom I have been speaking, will teach us a lot about how to make this work at Government scale.

July 14th, 2009

New open source Semantic Web store from Garlik capable of enterprise scale

Posted by Paul Miller @ 5:20 am

Categories: Podcasts, Semantic Web, Semantic Web Companies, Talking Semantics

Tags: Open Source, Semantic Web, RDF, XML, Internet, Software/Web Development, Web Development, Paul Miller

An oft-repeated concern in discussing large-scale deployment of Semantic Web ideas is that of ’scale.’ With many of the better known data stores upon which the Semantic Web depends capable of storing only tens or at best a few hundreds of millions of RDF triples, it can be difficult to argue that the technology is fit for real-world deployment at scale.

There are, of course, different ways of managing data, and it’s not always necessary to store everything in one massive store… but for those concerned about scale today’s news from UK-based Garlik may well put their minds at rest.

The company has taken their internally developed (and massively scalable) RDF triple store and released it to the world under an Open Source license as 4store.

I spoke with the company’s CEO and Head of Architecture just ahead of the launch, to learn more about the system and their motivation behind sharing it.

The result has just been released as a podcast.

July 9th, 2009

Semantic Web Gang podcast looks back at the Semantic Technology Conference

Posted by Paul Miller @ 4:48 am

Categories: Podcasts, Semantic Web, Semantic Web Gang

Tags: Conference, Semantic Web, Podcasts, Internet, Paul Miller

June’s episode of the regular Semantic Web Gang podcast was recorded on stage at the Semantic Technology Conference in San Jose.

Audio and video of the session is now available, with Gang members and conference organiser Tony Shaw engaging in a discussion of the event’s highlights and the underlying trends at work.

June 18th, 2009

New York Times embraces Linked Data

Posted by Paul Miller @ 12:36 pm

Categories: Open Data, Semantic Web

Tags:

The keynote on this final day of the Semantic Technology Conference saw Robert Larson and Evan Sandhaus of the New York Times talk about the paper’s innovative adoption of semantic technologies;

“The first semantic search system for The New Times was released in 1913 and was available bound in either paper ($6) or cloth ($8). In the 96 years since the advent of The Historical Index to The New York Times, semantic technology has become central to The New York Times’ daily operations and the focus of much internal research and development. In our keynote, Rob Larson, VP of Digital Production, and Evan Sandhaus, Semantic Technologist, will review the long history of semantic technology at The New York Times; discuss the application of this technology in our operations; and review an innovative initiative to enlist the global community in solving some of our toughest challenges.”

Sandhaus and Larson begin by referring back to the Times‘ long history, and the early importance of the paper’s emphasis on building - and selling - a comprehensive abstracting and indexing service to stories in the paper. This, they suggest, was important in leading to the paper being considered as the paper of record, ahead of its numerous competitors.

Building upon the paper’s nine-month old ‘Annotated Corpus’ and its associated APIs, Larson closed the session by announcing that the Times‘ thesaurus is to be made available using a license and APIs that will see it available to play a part in the wider Linked Data cloud.

June 17th, 2009

Nova Spivack interviews Wolfram Alpha's Russell Foltz-Smith

Posted by Paul Miller @ 1:26 pm

Categories: Commercialisation, Semantic Web, Semantic Web Companies, Semantic Web People

Tags:

Radar Networks attracted a fair degree of attention with their roll-out of Twine, and the company’s CEO has built a reputation as one of the more thoughtful thinkers in the space. Nova took to the stage at the Semantic Technology Conference today, not to talk about his own company or ideas, but to lead a conversation with Russell Foltz-Smith from Wolfram Research.

Wolfram Research, of course, is the company behind the recently launched Wolfram Alpha; a ‘computational knowledge engine’ that attracted a wave of attention that reached into the mainstream media.

“Putting all of the world’s computable knowledge; it sounds impossible… or over-confident, maybe. What is computable knowledge?”

“It’s ’systematic knowledge.’ It can be compared, contrasted, correlated, computed on. It’s not a movie review. Examples are classical physics, financial data and models, weather data and models… It’s not the latest opinion on who Britney Spears is dating. We don’t have a model to do anything with that in our system.”

Nova asks if it’s the difference between objective and subjective… Alpha deals with objective information. ‘Facts,’ almost?

Nova asks about sources, pointing to the example of Tibet; is it a ‘fact’ that Tibet that is part of China, or not… ?

“In the case of geo-political things, and religious things, we have to make choices… and allow the community to let us know whether they agree or not…” Couldn’t the system represent multiple views, tied to the diverse sources? Could we not show the different opinions, and allow the user to make informed decisions themselves?

Nova; “is the world’s computable knowledge infinite?”

Russell; “the foundation of computable knowledge is likely to be finite… The amount of knowledge that can be computed and generated from that is infinite…”

Nova; “I can see that maths could be finite. But geopolitics, health, etc… that’s much, much larger…”

Russell; “The instances seem very complex… Huge, but finite… I don’t want anyone to think we’ll have this done in ten years… It’s a long term thing.”

Nova; “Stephen [Wolfram] reckons it could be done in three years…?”

Nova; “Looking at the back end, the ontology seems to be implicit. I didn’t see any classes, just a lot of instances… a set of facts. As the team grows, how do you prevent people adding facts in different ways?”

Russell; “There are a set of stored facts; things you know about a city. But then there are computed facts that you couldn’t store in a traditional ontology.” Huh?

Nova; “Can you make a statement about what percent of the world’s computable knowledge is there today?”

Russell; “I can’t make a statement…”

Nova; “The syntax is quite interesting… but enigmatic. It wasn’t necessarily that the knowledge wasn’t there, but that I’d asked for it in the wrong way. Can’t you make a manual? … Stephen [Wolfram] said it would be an impossible task to write the manual… or to make a generic natural language on top.”

Nova; “In some cases a naive query will get you the answer, but maybe there’s a need for a layer that helps you when you don’t get what you want…”

Russell; “I think we’re getting close… we’re going to put an API out in the next few weeks, and hopefully someone will build the application using that to parse natural language and translate it for Alpha… Do we spend our time doing that, or putting more data and more models into the system… I reckon our time is best spent adding more data…”

Nova; “Is there a set of schemas or ontologies to link all of this stuff together?”

Russell; “There isn’t an ontology over the whole system… but within a domain there is structure… Is there some grand scheme that we have internally? Not really. The company has been doing this stuff for 23 years, so there’s a bit of a shared understanding internally.”

Examples keep coming back to mathematics… To succeed, Alpha has to offer compelling examples that are far broader…

Nova; “What about reasoning. You’ve said that you can derive additional knowledge. What kinds of reasoning is the system capable of?”

Russell; “I’d call it very simple reasoning. For example changing the currency based upon your geo-location… Is there any weak or strong AI in here? Not really. Could you build something like that? Probably. Will we? I don’t know…”

Nova; “Alpha seems to be a subset of Mathematica capabilities… Would you expand that, and bring a full Mathematica to the Web”

Russell; “It is, and there are plans to extend the capabilities. I don’t know if we’d go to a full-blown Mathematica on the web.”

Russell’s mentioning a subscription service for people working with more data, or needing more compute time. The public web site tends to time out a query in 4-8 (or 48?) seconds… The professional subscription version will have a monthly subscription version that will allow you to compute bigger questions. There will also be a pay-per-use API… and ‘primitive’ advertising. More advanced advertising, based on transactions, to be launched soon.

Nova; “Alpha’s really cool, but I want to do this on my own knowledge… inside an enterprise, inside a government agency…”

Russell; “We can roll out a custom Wolfram Alpha for those who want it behind the firewall. We will also let people upload their own data sets. We need to find a sensible way to let people do this…”

Nova; “There was a lot of hype - possibly my fault - around Alpha being a Google Killer. Obviously it’s not that. It’s something quite different. Who is the user, and what are they using it for?”

Russell; “Use will evolve, and it already has. There’s an obvious use by students, but the school year has just ended.

Nova; “Wolfram Alpha; now even Ph.D’s can cheat on their homework.”

Nova; “Are consumers using it? Obviously they’re having a play, but are they coming back and using it?”

Russell moves off to talk about academic use… Dodging the question?

Nova; “Are the financial capabilities in Alpha differentiated from the capabilities banks and investors already have in their vertical?”

Russell; “more sophisticated than a general finance web site, but probably less sophisticated than you’d find on a terminal in a bank.”

Nova; “Do I really need to know how long it would take an ant to get from San Francisco to Cairo?”

Russell; “Because of the way the system is engineered, it just keeps computing until it runs out of time. With simple queries you’ll get a lot of data. It just keeps computing.”

Nova; “What’s the big challenge, moving forward?”

Russell; “Setting priorities.”

Nova; “So let’s talk about Google. They made some aggressive marketing moves during the Alpha roll-out, and they’re continuing to roll products out to chip away… Do you think that what you’ve built is defensible, just because it’s hard… or can you defend it in other ways?”

Russell; “There are significant barriers to what we’re doing. Someone else could build this… but would they want to? That’s an open question.”

Nova; “Do you hope to work with other companies? Perhaps revenue share with them?”

Russell; “Obviously.”

Nova; “There’s been a lot of interest in how Alpha might connect with open standards and the Semantic Web…”

Russell; “If you want the platform to be used, we’ll have to do some of this stuff… RDF, OWL, etc could play a huge role.”

Nova; “Timeframe?”

Russell; “It’ll depend on pick-up of the API… which is due out in a few weeks.”

Nova; “So what’s the implication for education? It makes it possible to do some things without even thinking…”

Russell; “It’ll be a heated debate for a while… Some things are positive, some negative. There’s going to be a reorientation… It has to happen.”

Nova; “The danger is that if you delegate thinking [inside education] to a computation service… you may not actually understand enough to know if the answer that comes back is correct.”

Russell; “That’s a valid concern.”

Q&A

“You rely more on your computational engine than natural language… but you lay a lot of emphasis on the linguistics in your system. So if it’s not NLP what is it?”

Russell; “Domain linguistics, mainly; mathematical language, engineering language, etc… We think about how people describe things and search in these domains… and crawl the web looking for examples of how people use language in these domains.”

“Stephen is focussed on quality of data, which is important to a lot of people here. There aren’t a lot of tools. In addition to making your data store, I wonder if there might be scope to make some of your data curation tools available to the community, to improve the data out there.”

Russell; “Great point. Can we make these tools genuinely useful to people, without creating a support nightmare…”

June 17th, 2009

Semantic Search Round Table at the Semantic Technology Conference

Posted by Paul Miller @ 9:40 am

Categories: Research, Semantic Web, Semantic Web Companies, Web 3.0

Tags:

Wednesday’s opening Keynote here in San Jose sees Guidewire’s Carla Thompson joined on stage by senior representatives from many of the more interesting players in the Semantic Search space; Tomasz Imielinski from Ask, Peter Norvig from Google, Riza Berkan of Hakia, Scott Provost from Microsoft, William Tunstall-Pedoe of the UK’s True Knowledge, and Andrew Tomkins of Yahoo.

Carla asks each panellist to describe the differentiating aspects of their product in ‘one or two sentences;’

Tomasz; “we receive about three times as many questions as other search companies. We want to answer questions the best we can from multiple sources… using structured and unstructured data.”

Scott; “Bing really focusses on understanding the intent behind queries, and organising the page to help people get to their answer much faster.”

Peter; “We focus on being comprehensive, accurate and fast… so we have to keep on innovating in crawling, ranking, systems engineering. One thing that differentiates us… most companies decide whether to focus on marketing or sales. We focus on engineering.”

Riza; “We are a complete semantic search engine, from the bottom up. We don’t even have an index. We’ve optimised the entire process for semantic operations. We focus on credible and dynamic content, and offer users a new perspective.” Instead of popularity, they focus on credibility.

William; “True Knowledge is a platform that does direct question answering. There’s a knowledge base and an inference engine to answer questions we haven’t seen before.” True Knowledge tries to ‘help when it can, and stay quiet when it can’t,’ as can be seen demonstrated in their recently released Firefox plugin.

Andrew: “Yahoo! is very aggressive about semantic annotation… SearchMonkey is about acquiring semantic information and surfacing it in search results on the page.”

Carla mentions Tom Tague’s keynote from yesterday, where he suggested that ’semantic search is an answer to a question no one is asking’… so “why do we need to change search?”

Tomasz responds, suggesting that users don’t necessarily demand new products that subsequently become successful. eg; no one was asking for the iPod before it launched. “When they see it, they will want it.”

Turning to Google and Yahoo!, Carla asks them “why do we need to change search?”

Peter… “as an industry, satisfaction is very high… but that is just because that’s what people know [now]… People don’t like technology… people like solutions. When we deliver it, people will want it.”

Andrew; “Does search need to change? It already is… Today, on any major search engine, if you search for a restaurant, you’ll see structured information about that restaurant; reviews, phone number, etc… This has been accelerating over the last 3-4 years… When we put this information up, and trigger it correctly, we see far higher levels of engagement from our users than anything else.”

Carla; “it may be a stupid question, but it has to be asked; what is semantic search?”

Scott; “it means a lot of different things. At Powerset we focussed on understanding the meaning in web pages, so we could present them, rank them…”

Carla; “Has Powerset’s focus been diluted by the [Microsoft] acquisition?”

Scott; “No.”

Carla asks Riza; “Someone from Hakia that I spoke to last year said you were the only one doing ‘true semantic search.’ Is that true?”

Riza; “No… Semantic Search can enrich search results… Semantic Search can improve precision/disambiguation… Semantic Search can organise results better. In the future, search will move to more conversational systems, and for that you really need semantic technology.”

Carla; “How do you measure the ’semanticity’ of a search engine?”

Tomasz; “That’s my favourite question… We took a sample of ‘equivalent’ queries from the logs, and ran it to evaluate ranking etc; does the search engine give similar answers to questions like ‘Top 10 songs’ and ‘Top Ten songs,’ etc. Should they?”

Andrew; “It’s incredibly hard to understand what a user will like… if you mess with the logo, it changes the perception of the results… if you make tiny changes, it can have a big impact on perception… When it comes to understanding semantic contact in search, we should identify the task the user is trying to solve… and have a metric that’s aligned to that use case… We can break search queries today into different classes; how do we do when a user is trying to book dinner, or a vacation? Semantic Technology should be judged on its impact based on these task metrics rather than any underlying notions of entity resolution, etc… SearchMonkey, for instance, lets users inject structured data into the process… The information can be incorporated in any way… and change how the results are presented. We have about 15,000 people in our development community, changing the way those results are presented every day.”

Tomasz; “I would expect a semantic search engine to deliver equivalent results to queries that would appear similar to a human being; ‘Top 10 songs’ and ‘Top Ten Songs’ should deliver the same answer. Today in most mainstream search engines they don’t.”

Carla; “Search v. Answers. True Knowledge is billed as ‘the Internet Answer Engine;’ is it necessary to move search to an answer-based format, or has Google trained users to think in keywords?”

William; “We support both keyword search and full-text questions. It’s important to answer users’ questions.”

Peter; “Different types of answers are appropriate for different types of questions; sometimes the answer is a fact, or a page, or a series of results to support a process of study. To say there’s going to be one technology or one type of answer doesn’t make sense.”

Riza; “You could be asking a ‘where,’ ‘why,’ ‘how’ type of question. Questions are important, and the search engine needs to be able to interpret the mode of the question and return results appropriately.”

Carla; “You mentioned talking about the credibility of search results. How do you define a ‘credible’ search result, and how much of a need is there really? I’m not hearing users question the credibility of search results they see today.”

Riza; “Practically, credibility is important in ’serious’ subjects; medical information, etc. You want to know where the results come from and how credible they are. When it comes to credible content, you can’t really do a statistical search or have a ‘popularity vote,’ because much credible content isn’t ‘popular.’

Scott; “People’s expectations for credibility are different depending upon the query. If you ask an ‘instant answer’ type query you expect the answer to be credible. If you do a broader search, you expect a mix of results to be returned”

William; “If a system understands structured knowledge, it can understand when different sources contradict one another”

Riza; “A system doesn’t need to know what’s credible; we can go to a librarian for that. Hakia doesn’t decide whether a resource is credible or not; we use librarians for that”

Tomasz; “If you ask for the capital of Japan we expect a single answer. If you ask about taxes, maybe the IRS is the best source but there are others. If you ask ‘how to get rid of acne’ you expect a lot of results.”

Carla; “We’ve seen three news-making launches in the past month; Wolfram Alpha, Bing, Siri. Is Wolfram the first step towards 2001? How is this engine valuable to those of us who don’t need to solve complex maths?”

Scott; “it’s not the first step… we’ve been working on these problems for a long time. There are a lot of questions people want to ask about the types of data that Wolfram aggregates… We see these things as part of full-search services. Powerset has moved along this path as well, pulling structured data in response to full-text queries.”

William; “Wolfram is a tremendous effort. An interesting example of question answering with structured data. I think people will find uses for it in particular use cases; I spoke to someone who’d used it to calculate when his visa expired, because it could do date calculation. I think there will be use cases in various scenarios; maths, nutrition information, etc… if you remember that it has that sort of information and remember to go to it… However one thing it doesn’t have is a decent back-fill. If it doesn’t have the data, or doesn’t understand the way you asked the query, it gives you nothing. We try to keep quiet and fail over to standard internet search in that sort of circumstance.”

Carla; “Does a semantic search engine know how not to answer a question?”

William; “that’s absolutely fundamental. You need the ability to reliably keep quiet when you don’t have the answer… and fail over reliably to other search services. [True Knowledge does try to do this...] “That requires very high quality semantics.”

Andrew; “One way to characterise the approach of Wolfram Alpha is that it’s a centralised approach. The Wolfram Alpha team goes out to find data and bring it in-house to convert to a standard form. A different approach is to have an ecosystem contributing data in the public eye… It’s not clear yet how much of a value-add is going to come from this centralised knowledge mapping approach. Yahoo! is focussed on the ecosystem approach, and helping people with knowledge to make it available.”

Peter; “Our inclination would be that we don’t want a closed walled garden. We want all the information available to combine in different ways. We want the information to be open, and the tool set to be open for mashing up in different ways.”

Scott; “If Wolfram Alpha hadn’t taken a walled garden approach they might never have launched a product.”

Tomasz; “Wolfram Alpha is great, but it’s not a search engine”

Carla; “Siri… caused a lot of buzz, uses True Knowledge… what are your thoughts?”

Andrew; “To be counter-cultural… the notion of getting much deeper and assisting a user with a task is spot on. We’re going to see much more of that. Search has tended to be stateless. Each query you enter is more or less processed without context. Yahoo! is rolling out more stateful search tools, and other companies will do the same. We expect people to use these tools on lots of devices. Would be expect people to come to the same place for purchase, navigation, etc? Do we expect one interface? There are going to be virtual assistants… I just don’t know if they’re going to be embedded into a search box.”

Scott; “Conversation is the ultimate user interface… but it’s not clear that I want to have a conversation with my laptop during the working day. How do I display the results? But there’s a huge role for conversation and dialogue in refining search and getting a user to their results faster.”

Tomasz; “What is the goal of Siri? If you try to go to broad you become a search engine.”

Scott; “When people have a conversational interface, they won’t speak in keywords.”

Carla; “What are the larger goals for Bing?”

Scott; “Bing is trying to simplify key tasks that people do when they come to a search engine. In travel, health, shopping, we can understand what people are trying to do, and get them to better results faster. The thinking has evolved from ten blue links to the whole page, and organising things to help the user by understanding their tasks.”

Carla; “Peter; what did you think of Bing?”

Peter; “I like the idea of innovation in the user interface. There’s a lot of room for that. There’s been a lot of emphasis on getting the ranking right. You still need to do that, but other things are important too. I’m usually happy with results on my big screen. On a mobile device, I’m usually not happy with the results I get.”

June 16th, 2009

Semantic Technology Conference kicks off with Keynotes from Open Calais and Siri

Posted by Paul Miller @ 9:54 am

Categories: Commercialisation, Open Data, Semantic Web, Semantic Web Companies, Web 2.0, Web 3.0

Tags: Web, Advertisement, Tool, Siri, Tom Tague, Tague, Enterprise Adoption, Virtual Personal Assistant, Virtual Personal Assistant Paradigm, Podcasts

This year’s Semantic Technology Conference got fully underway this morning, with Keynote presentations from Tom Tague of Thomson Reuters’ Open Calais Initiative and Tom Gruber from Siri.

Despite the wider economic situation, attendance for this fifth year of the event feels a little up on last year, and there’s clearly real enthusiasm in the buzzing Halls.

Tague’s Open Calais has been one of the success stories for useful and easy application of semantic technologies beyond a core community of enthusiasts and adopters, and has been covered here and on Cloud of Data a number of times since it launched. Just today, they announced a new set of partners and a postal service that should remove one more perceived barrier for another set of potential adopters.

Speaking to the theme of ‘Web 3.0 - the Web of Me,’ Tague’s abstract suggests;

“The mainstream adoption of Web 2.0 technologies – from RSS feeds to social networks – is hastening the demise of the portal. With each new face on Facebook, and each new Twitter account, our once routine habits and traffic patterns shift. This wave of change in the way we consume, transact and interact on the Web is dis-intermediating ‘destination’ sites of all kinds. Our once centralized content has been atomized.

And yet our fundamental problem persists. We’re overwhelmed with input, yet still can’t find the one thing we need… now.

Semantic technologies – and the content interoperability and Linked Data connections they beget – offer new hope. That is not to say the answer lies in building new search engines, and few would argue for another news aggregator. Rather, our point of inflection lies at the point of consumption. Our task is to simultaneously refine and enrich our digital experience of everything from content and community to commerce.”

Early on, Tague made a ‘non-apologetic statement;’

“People need to start deriving financial benefits from semantic technology. It’s time”

Absolutely!

Tague looks back at the move from ‘Web 1.0,’ described as ‘the last Web we agreed on,’ to ‘Web 2.0,’ which he sees as largely defined by the ‘addition of social.’ Today, he reckons, we are ‘extraordinarily content-rich’, ‘extraordinarily information-poor’ and ‘experientially deficient.’ Despite a wealth of content, we are failing to make the most of it.

‘We’re at the inflection point’ where ‘innovation is exploding’ as we move from developing and inventing toward mainstream adoption of technologies in the semantic technology space. Lots of things will be tried; 90% will fail, but that’s ok.

‘Everyone needs plumbing,’ and that’s what Calais is; semantic plumbing. 13 version releases in 18 months; about 100 presentations, 13,000 registered Open Calais developers, a million great ideas.

Tague reckons the various efforts he comes in contact with fall into six broad buckets;

Tools; Social; Advertising; Search; Publishing; Interface.

First, Enabling Tools. Data Management, Data generation, Databases, Integration and workflow. ‘A big yes.’ ‘We need tools.’ Everyone needs tools, especially as you move from early adopters toward the mainstream. Tools build the bridges that cross the chasm to enterprise adoption.

Enterprise adoption will not happen because it’s cool. Enterprise adoption will not be talked about on Twitter. Enterprise adoption will happen because it’s cheaper/faster/better than what they have just now.

‘Tool vendors need to simplify their story; it’s not about more functionality.’ ‘If I can’t understand your story, then Enterprise IT certainly can’t’

Second, ‘let’s put some frosting on top of social.’ ‘Wouldn’t it be cool if we could…’ Some of it might be cool, but there’s a challenge in monetising social. Adding frosting to the top of an industry that hasn’t worked out its own monetisation is fraught with risk.

‘I haven’t seen a compelling story yet.’

Next, advertising. Almost a dirty word in the semantic technology domain last year. But advertising is fuel, and semantic technologies have a clear role to play in enhancing advertising (see my podcast with Scott Brinker from last year…).

Semantic search; ‘the semantic industry’s brilliant yet under-achieving child.’ The answer to a question no one is asking? General, consumer-facing semantic search… directly competing with Google et al? Not viable.

But vertical search in specific domains… a huge growth opportunity, and people are willing to invest the time, effort and money to make it happen. Room for a handful of players in each domain?

Search; ‘a bifurcated marketplace.’

Publishing; content producers, editorial/aggregation, ‘robotic publishing.’

‘Classic publishers can get enormous value from this technology… not all of the value is in the user experience.’ Much of the value is being found in the back office, making existing data and investments work harder.

Little value in ‘robotic publishing,’ because the content isn’t that readable. Aggregation services like Huffington Post and Daily Me present ‘enormous opportunities.’

Interface; gaming a huge and growing market. $57bn industry. A ’seamless, interactive and responsive experience,’ it’s ‘graphically engaging and fun.’

Zemanta, AdaptiveBlue, Feedly, Apture et al ‘trying to make the consumption experience different’ [better?]. Not suggesting that these are like a game, but many of the drivers may be similar?

“People are on their mobile devices and in the browser; go where the people are.” Which links well to the next keynote…

“Do you care about semantics or about user value?”

“Don’t fund/buy semantic infrastructure beyond what you need; use infrastructure built by others where possible.”

“Think very hard about the user experience; make it compelling and exciting.”

Following Tague’s presentation, Tom Gruber took to the stage to talk about Siri; a company building a Virtual Personal Assistant (with an interesting iPhone app to start things off) that we discussed during a podcast last week. As Gruber’s says;

“We are beginning to see a new interaction paradigm for the web: the Virtual Personal Assistant (VPA). A VPA is task focused: it helps you get things done. You interact with it in natural language, in a conversation. It gets to know you, acts on your behalf, and gets better with time. The VPA paradigm builds on the information and services of the web, with new technical challenges of semantic intent understanding, context awareness, service delegation, and mass personalization.

Siri is a virtual personal assistant for the mobile Internet. Although just in its infancy, Siri can help with some common tasks that human assistants do, such as booking a restaurant, getting tickets to a show, and inviting a friend. We will describe the technology underlying Siri and how it fits in the larger ecosystem of services and data providers. And we will offer a vision of where assistants like Siri are going.”

Tom starts off by showing the Knowledge Navigator video from Apple… which dates all the way back to 1987. Many of the ideas are now coming to fruition; touch screens, a global network, awareness of temporal and social context, speech in and out, a ‘conversational interface,’ ‘delegation of work’ to the machine, and trusted use of personal data.

Is the Knowledge Navigator possible today? ‘No, but we’re getting there.’

Siri is pretty close… in certain well understood contexts, as Gruber shows in a video demo of the evolving iPhone application.

What is a Virtual Personal Assistant? It does things for you; it’s task-oriented. It understands your intent via a conversational metaphor. It gets to know you; it’s not the same for everybody, unlike a search engine.

‘Service delegation [like Siri]; the mother of all mashups’

‘Context is king’ in communicating with a VPA; where am I, what time is it, who am I, etc.

“This really is the beginning of the age of the start of Virtual Assistants.”

Need to solve authorisation/ authentication. If we reach a ‘data commons’ there will be more, better, information to drive choices and decisions.

Tom Tague is a regular member of the Semantic Web Gang podcast, which I moderate. Tom Gruber was the latest guest in my Executive Briefing podcast series.

May 29th, 2009

Bing is not alone; similar techniques alive and well in existing vertical search

Posted by Paul Miller @ 4:10 am

Categories: Commercialisation, Podcasts, Semantic Web Companies, Talking Semantics

Tags: Endeca Technologies Inc., Technique, Robin, Podcasts, Productivity, Search, Internet, Paul Miller

Microsoft’s Bing is attracting plenty of interest today, and perhaps deservedly so as it brings some interesting fresh ideas to the world of generic search engines. Whether it is sufficiently compelling to break our deeply ingrained association of ’search’ with ‘Google’ remains to be seen.

It should be remembered, of course, that broadly similar approaches are already taken to managing and navigating data inside the data centres of large corporations where Autonomy, FAST, Endeca and their peers provide powerful capabilities.

I recorded a podcast with Endeca Chief Scientist Daniel Tunkelang in January and, by chance, spoke with Robin Johnson yesterday. Robin is CEO of FT Search, part of the Financial Times Group, and responsible for a new vertical search tool called Newssift. Newssift combines components from various technology companies (including Endeca, Nstein, Lexalytics and ReelTwo) to offer a useful means of learning more about businesses and the external factors affecting them.

May 22nd, 2009

Semantic Web Gang podcast discusses Wolfram Alpha and Google's Rich Snippets

Posted by Paul Miller @ 6:14 am

Categories: Podcasts, Semantic Web Gang

Tags: Google Inc., Podcast, Mainstream Media, Semantic Web, Internet, Paul Miller

This month has seen Google announce ‘Rich Snippets‘ and Wolfram Research release Alpha to a flurry of mainstream media coverage; both are of interest to those working on the Semantic Web.

This month’s episode of the Semantic Web Gang takes a look at both stories, and Gang members share their impressions on the news and what it might mean moving forward.

Next month’s Semantic Web Gang will be coming live from the Semantic Technology Conference in San Jose.