1. Identity,
2. Access,
3. Storage,
4. Representation, and
5. Presentation.

I ended up with what I can best describe as the Data 3.0 Manifesto. A manifesto for standards complaint access to structured data object (or entity) descriptors.

Some Related Work

Alex James (Program Manager Entity Frameworks at Microsoft), put together something quite similar to this via his Base4 blog (around the Web 2.0 bootstrap time), sadly -- quoting Alex -- that post has gone where discontinued blogs and their host platforms go (deep deep irony here).

It's also important to note that this manifesto is also a variant of the TimBL's Linked Data Design Issues meme re. Linked Data, but totally decoupled from RDF (data representation formats aspect) and SPARQL which -- in my world view -- remain implementation details.

Data 3.0 manifesto

• An "Entity" is the "Referent" of an "Identifier."
• An "Identifier" SHOULD provide a global, unambiguous, and unchanging (though it MAY be opaque!) "Name" for its "Referent".
• A "Referent" MAY have many "Identifiers" (Names), but each "Identifier" MUST have only one "Referent".
• Structured Entity Descriptions SHOULD be based on the Entity-Attribute-Value (EAV) Data Model, and SHOULD therefore take the form of one or more 3-tuples (triples), each comprised of:
  • an "Identifier" that names an "Entity" (i.e., Entity Name),
  • an "Identifier" that names an "Attribute" (i.e., Attribute Name), and
  • an "Attribute Value", which may be an "Identifier" or a "Literal".
• Structured Descriptions SHOULD be CARRIED by "Descriptor Documents" (i.e., purpose specific documents where Entity Identifiers, Attribute Identifiers, and Attribute Values are clearly discernible by the document's intended consumers, e.g., humans or machines).
• Structured Descriptor Documents can contain (carry) several Structured Entity Descriptions
• Stuctured Descriptor Documents SHOULD be network accessible via network addresses (e.g., HTTP URLs when dealing with HTTP-based Networks).
• An Identifier SHOULD resolve (de-reference) to a Structured Representation of the Referent's Structured Description.

Related

• Referent, Identifier, and Descriptor/Sense (The Data Perception Trinity) illustration
• Referent, Identifier, and Descriptor/Sense Trinity (as exploited in FOAF+SSL based Secure WebIDs) illustration
• Demystifying Linked Data via EAV Model based Structured Descriptions
• What do people have against URIs and URLs?
• The URI, URL, and Linked Data Meme's Generic HTTP URI
• Simple Explanation of RDF and Linked Data Dynamics
• Linked Data and Identity
• FOAF+SSL FAQ
• LOD Community Thread (showing evolution of this manifesto based on feedback from members such as Richard Cyganiak).
• Googlebase Data API Docs
• Google Data Protocol (GData)
• Microsoft's OData Protocol
• Magic of De-referencable Names and actual Data via Binky Video
• Social Objects Presentation (aka. Social Linked Data Objects) - by Jyri Engeström
• What's a Reference?

At the end of the marathon session, it was clear to me that a blog post was required for future reference, at the very least :-)

What is Linked Data?

"Data Access by Reference" mechanism for Data Objects (or Entities) on HTTP networks. It enables you to Identify a Data Object and Access its structured Data Representation via a single Generic HTTP scheme based Identifier (HTTP URI). Data Object representation formats may vary; but in all cases, they are hypermedia oriented, fully structured, and negotiable within the context of a client-server message exchange.

Why is it Important?

Information makes the world tick!

Information doesn't exist without data to contextualize.

Information is inaccessible without a projection (presentation) medium.

All information (without exception, when produced by humans) is subjective. Thus, to truly maximize the innate heterogeneity of collective human intelligence, loose coupling of our information and associated data sources is imperative.

How is Linked Data Delivered?

Linked Data is exposed to HTTP networks (e.g. World Wide Web) via hypermedia resources bearing structured representations of data object descriptions. Remember, you have a single Identifier abstraction (generic HTTP URI) that embodies: Data Object Name and Data Representation Location (aka URL).

How are Linked Data Object Representations Structured?

A structured representation of data exists when an Entity (Datum), its Attributes, and its Attribute Values are clearly discernible. In the case of a Linked Data Object, structured descriptions take the form of a hypermedia based Entity-Attribute-Value (EAV) graph pictorial -- where each Entity, its Attributes, and its Attribute Values (optionally) are identified using Generic HTTP URIs.

Examples of structured data representation formats (content types) associated with Linked Data Objects include:

• text/html
• text/turtle
• text/n3
• application/json
• application/rdf+xml
• Others

How Do I Create Linked Data oriented Hypermedia Resources?

You markup resources by expressing distinct entity-attribute-value statements (basically these a 3-tuple records) using a variety of notations:

• (X)HTML+RDFa,
• JSON,
• Turtle,
• N3,
• TriX,
• TriG,
• RDF/XML, and
• Others (for instance you can use Atom data format extensions to model EAV graph as per OData initiative from Microsoft).

You can achieve this task using any of the following approaches:

• Notepad
• WYSIWYG Editor
• Transformation of Database Records via Middleware
• Transformation of XML based Web Services output via Middleware
• Transformation of other Hypermedia Resources via Middleware
• Transformation of non Hypermedia Resources via Middleware
• Use a platform that delivers all of the above.

Practical Examples of Linked Data Objects Enable

• Describe Who You Are, What You Offer, and What You Need via your structured profile, then leave your HTTP network to perform the REST (serendipitous discovery of relevant things)
• Identify (via map overlay) all items of interest based on a 2km+ radious of my current location (this could include vendor offerings or services sought by existing or future customers)
• Share the latest and greatest family photos with family members *only* without forcing them to signup for Yet Another Web 2.0 service or Social Network
• No repetitive signup and username and password based login sequences per Web 2.0 or Mobile Application combo
• Going beyond imprecise Keyword Search to the new frontier of Precision Find - Example, Find Data Objects associated with the keywords: Tiger, while enabling the seeker disambiguate across the "Who", "What", "Where", "When" dimensions (with negation capability)
• Determine how two Data Objects are Connected - person to person, person to subject matter etc. (LinkedIn outside the walled garden)
• Use any resource address (e.g blog or bookmark URL) as the conduit into a Data Object mesh that exposes all associated Entities and their social network relationships
• Apply patterns (social dimensions) above to traditional enterprise data sources in combination (optionally) with external data without compromising security etc.

How Do OpenLink Software Products Enable Linked Data Exploitation?

Our data access middleware heritage (which spans 16+ years) has enabled us to assemble a rich portfolio of coherently integrated products that enable cost-effective evaluation and utilization of Linked Data, without writing a single line of code, or exposing you to the hidden, but extensive admin and configuration costs. Post installation, the benefits of Linked Data simply materialize (along the lines described above).

Our main Linked Data oriented products include:

• OpenLink Data Explorer -- visualizes Linked Data or Linked Data transformed "on the fly" from hypermedia and non hypermedia data sources
• URIBurner -- a "deceptively simple" solution that enables the generation of Linked Data "on the fly" from a broad collection of data sources and resource types
• OpenLink Data Spaces -- a platform for enterprises and individuals that enhances distributed collaboration via Linked Data driven virtualization of data across its native and/or 3rd party content manager for: Blogs, Wikis, Shared Bookmarks, Discussion Forums, Social Networks etc
• OpenLink Virtuoso -- a secure and high-performance native hybrid data server (Relational, RDF-Graph, Document models) that includes in-built Linked Data transformation middleware (aka. Sponger).

Related

• Hypertext Transfer Protocol 1.1 RFC
• Open Data Protocol Glossary
• Simple Explanation of RDF and Linked Data Dynamics
• Collection of post from the past about Linked Data
• Are We There Yet Re. Web++? -- includes link to podcast conversation with Jon Udell
• Web of Linked Data Pivoting Demo from TED -- by Microsoft's Gary Flake
• Microsoft Pivot atop Virtuoso Quad Store's Faceted Browser Engine-- My Demonstration of EAV model transcending data representation variations (i.e., RDF's EAV data model data served up in Microsoft CXML data representation format).

Since the beginning of the modern IT era, each period of innovation has inadvertently introduced its fair share of Data Silos. The driving force behind this anomaly remains an overemphasis on the role of applications when selecting problem solutions. Unfortunately, most solution selecting decision makers remain oblivious to the fact that most applications are architecturally monolithic; i.e., they fail to separate the following five layers that are critical to all solutions:

1. Data Unit (Datum or Data Object) Identity,
2. Data Storage/Persistence,
3. Data Access,
4. Data Representation, and
5. Data Presentation/Visualization.

The rise of the Internet, and its exponentially-growing user-friendly enclave known as the World Wide Web, is bringing the intrinsic costs of the monolithic application architecture anomaly to bear -- in manners unanticipated by many. For example, the emergence of network-oriented solutions across the realms of Enterprise 2.0-based Collaboration and Web 2.0-based Software-as-a-Service (SaaS), combined with the overarching influence of Social Media, are producing more heterogeneously-structured and disparately-located data sources than people can effectively process.

As is often the case, a variety of problem and product monikers have emerged for the data access and integration challenges outlined above. Contemporary examples include Enterprise Information Integration, Master Data Management, and Data Virtualization. Labeling aside, the fundamental issues of the unresolved Data Integration challenge boil down to the following:

• Data Model Heterogeneity
• Data Quality (Cleanliness)
• Semantic Variance across Contexts (e.g., weights and measures).

Effectively solving today's data integration challenges requires a move away from monolithic application architecture to loosely-coupled, network-centric application architectures. Basically, we need a ubiquitous network-centric application protocol that lends itself to loosely-coupled across-the-wire orchestration of data interactions. In short, this will be what revitalizes the art of application development and deployment.

The World Wide Web is built around a network application protocol called HTTP. This protocol intrinsically separates the five layers listed earlier, thereby enabling:

• Use of Generic HTTP URIs as Data Object (Entity) Identifiers;
• Identifier Co-reference, such that multiple Data Object Identifiers may reference the same Data Object;
• Use of the Entity-Attribute-Value Model to describe Data Objects using real world modeling friendly conceptual graphs;
• Use of HTTP URLs to Identify Locations of Resources that bear (host) Data Object Descriptions (Representations);
• Data Access mechanism for retrieving Data Object Representations from persistent or transient storage locations.

What is Virtuoso?

A uniquely designed to address today's escalating Data Access and Integration challenges without compromising performance, security, or platform independence. At its core lies an unrivaled commitment to industry standards combined with unique technology innovation that transcends erstwhile distinct realms such as:

• Data Management (Relational, RDF Graph, or Document),
• Data Access Middleware,
• Web Application & Services Deployment,
• Linked Data Deployment, and
• Messaging.

When Virtuoso is installed and running, HTTP-based Data Objects are automatically created as a by-product of its powerful data virtualization, transcending data sources and data representation formats. The benefits of such power extend across profiles such as:

• Information & Knowledge Workers,
• Systems Integrators & Architects,
• Distributed Collaboration & Social Media,
• Cloud Computing, and
• Application Development.

Product Benefits Summary

• Enterprise Agility — Virtuoso lets you mix-&-match best-of-class combinations of Operating Systems, Programming Environments, Database Engines and Data-Access Middleware when building or tweaking your IS infrastructure, without the typical impedance of vendor-lock-in.
• Data Model Dexterity — By supporting multiple protocols and data models in a single product, Virtuoso protects you against costly vulnerabilities such as: perennial acquisition and accumulation of expensive data model specific DBMS products that still operate on the fundamental principle of: proprietary technology lock-in, at a time when heterogeneity continues to intrinsically define the information technology landscape.
• Cost-effectiveness — By providing a single point of access (and single-sign-on, SSO) to a plethora of Web 2.0-style social networks, Web Services, and Content Management Systems, and by using Data Object Identifiers as units of Data Virtualization that become the focal points of all data access, Virtuoso lowers the cost to exploit emerging frontiers such as socially-enhanced enterprise collaboration.
• Speed of Exploitation — Virtuoso provides the ability to rapidly assemble 360-degree conceptual views of data, across internal line-of-business application (CRM, ERP, ECM, HR, etc.) data and/or external data sources, whether these are unstructured, semi-structured, or fully structured.

Bottom line, Virtuoso delivers unrivaled flexibility and scalability, without compromising performance or security.

Related

• HTTP URI Abstraction and Linked Data
• Be The Master of Your Own Search Index
• Who's Data Is It?
• MDM & Linked Data
• What is Linked Data Oriented RDF-zation?
• Semantic Web: Travails to Harmony Illustrated

 

1. Acquire your own personal or service specific data space in the Cloud. Think DBase, Paradox, FoxPRO, Access of yore, but with the power of Oracle, Informix, Microsoft SQL Server etc.. using a Conceptual, as opposed to solely Logical, model based DBMS (i.e., a Hybrid DBMS Engine for: SQL, RDF, XML, and Full Text)
2. Ability to share and control access to your resources using innovations like FOAF+SSL, OpenID, and OAuth, all from one place
3. Construction of personal or organization based FOAF profiles in a matter of minutes; by simply creating a basic DBMS (or ODS application layer) account; and then using this profile to create strong links (references) to all your Data silos (esp. those from the Web 2.0 realm)
4. Load data sets from the LOD cloud or Sponge existing Web resources (i.e., on the fly data transformation to RDF model based Linked Data) and then use the combination to build powerful lookup services that enrich the value of URLs (think: Web addressable reports holding query results) that you publish
5. Bind all of the above to a domain that you own (e.g. a .Name domain) so that you have an attribution-friendly "authority" component for resource URLs and Entity URIs published from your Personal Linked Data Space on the Web (or private HTTP network).

In a nutshell, the AWS Cloud infrastructure simplifies the process of generating Federated presence on the Internet and/or World Wide Web. Remember, centralized networking models always end up creating data silos, in some context, ultimately! :-)

Naturally, I am not expecting everyone to agree with me. I am simply making my contribution to what will remain facinating discourse for a long time to come :-)

Related

• Web 3.0 The Best Official Definition Imaginable -- Nova Spivack's

Robin:

Web 3.0 is fundamentally about the World Wid Web becoming a structured database equipped with a formal data model (RDF which is a moniker for Entity-Attribute-Value with Classes & Relationships based Graph Model), query language, and a protocol for handling divrerse data representational requirements via negotiation

Web 3.0 is about a Web that facilitates serendipitous discovery of relevant things; thereby making serendipitous discovery quotient (SDQ), rather than search engine optimization (SEO), the critical success factor that drives how resources get published on the Web.

Personally, I believe we are on the cusp of a major industry inflection re. how we interact with data hosted in computing spaces. In a nutshell, the conceptual model interaction based on real-world entities such as people, places, and other things (including abstract subject matter) will usurp traditional logical model interaction based on rows and columns of typed and/or untyped literal values exemplified by relational data access and management systems.

Labels such as "Web 3.0", "Linked Data", and "Semantic Web", are simply about the aforementioned model transition playing out on the World Wide Web and across private Linked Data Webs such as Intranets & Extranets, as exemplified emergence of the "Master Data Management" label/buzzword.

What's the critical infrastructure supporting Web 3.0?

As was the case with Web Services re. Web 2.0, there is a critical piece of infrastructure driving the evolution in question, and in this case it comes down to the evolution of Hyperlinking.

We now have a new and complimentary variant of Hyperlinking commonly referred to as "Hyperdata" that now sits alongside "Hypertext". Hyperdata when used in conjunction with HTTP based URIs as Data Source Names (or Identifiers), delivers a potent and granular data access mechanism scoped down to the datum (object or record) level; which is much different from the document (record or entity container) level linkage that Hypertext accords.

In addition, the incorporation of HTTP into this new and enhanced granular Data Source Naming mechanism also addresses past challenges relating to separation of data, data representation, and data transmission protocols -- remember XDR woes familiar to all sockets level programmers -- courtesy of in-built content negotiation. Hence, via a simple HTTP GET --against a Data Source Name exposed by a Hyperdata link -- I can negotiate (from client or server sides) the exact representation of the description (entity-attribute-value graph) of an Entity / Data Object / Resource, dispatched by a data server.

For example, this is how a description of entity "Me" ends up being available in (X)HTML or RDF document representations (as you will observe when you click on that link to my Personal URI).

The foundation of what I describe above comes from:

1. Entity-Attribute-Value & Class Relationship Data Model (originating from LISP era with detours via the Object Database era. into the Triples approach in RDF)
2. Use of HTTP based Identifiers in the Entity ID construction process
3. SPARQL query language for the Data Model.

Some live examples from DBpedia:

• http://dbpedia.org/resource/Linked_Data
• http://dbpedia.org/resource/Hyperdata
• http://dbpedia.org/resource/Entity-attribute-value_model
• http://dbpedia.org/resource/Benjamin_Franklin

Related

• The End of RDBMS Primacy is Nigh
• Linking Open Data Community

A pre-installed edition of Virtuoso for Amazon's EC2 Cloud platform.

What does it offer?

1. Low cost entry point to a game-changing Web 3.0+ (and beyond) platform that combines SQL, RDF, XML, and Web Services functionality
2. Flexible variable cost model (courtesy of EC2 DevPay) tightly bound to revenue generated by your services
3. Delivers federated and/or centralized model flexibility for you SaaS based solutions
4. Simple entry point for developing and deploying sophisticated database driven applications (SQL or RDF Linked Data Web oriented)
5. Complete framework for exploiting OpenID, OAuth (including Role enhancements) that simplifies exploitation of these vital Identity and Data Access technologies
6. Easily implement RDF Linked Data based Mail, Blogging, Wikis, Bookmarks, Calendaring, Discussion Forums, Tagging, Social-Networking as Data Space (data containers) features of your application or service offering
7. Instant alleviation of challenges (e.g. service costs and agility) associated with Data Portability and Open Data Access across Web 2.0 data silos
8. LDAP integration for Intranet / Extranet style applications.

From the DBMS engine perspective it provides you with one or more pre-configured instances of Virtuoso that enable immediate exploitation of the following services:

1. RDF Database (a Quad Store with SPARQL & SPARUL Language & Protocol support)
2. SQL Database (with ODBC, JDBC, OLE-DB, ADO.NET, and XMLA driver access)
3. XML Database (XML Schema, XQuery/Xpath, XSLT, Full Text Indexing)
4. Full Text Indexing.

From a Middleware perspective it provides:

1. RDF Views (Wrappers / Semantic Covers) over SQL, XML, and other data sources accessible via SOAP or REST style Web Services
2. Sponger Service for converting non RDF information resources into RDF Linked Data "on the fly" via a large collection of pre-installed RDFizer Cartridges.

From the Web Server Platform perspective it provides an alternative to LAMP stack components such as MySQL and Apace by offering

1. HTTP Web Server
2. WebDAV Server
3. Web Application Server (includes PHP runtime hosting)
4. SOAP or REST style Web Services Deployment
5. RDF Linked Data Deployment
6. SPARQL (SPARQL Query Language) and SPARUL (SPARQL Update Language) endpoints
7. Virtuoso Hosted PHP packages for MediaWiki, Drupal, Wordpress, and phpBB3 (just install the relevant Virtuoso Distro. Package).

From the general System Administrator's perspective it provides:

1. Online Backups (Backup Set dispatched to S3 buckets, FTP, or HTTP/WebDAV server locations)
2. Synchronized Incremental Backups to Backup Set locations
3. Backup Restore from Backup Set location (without exiting to EC2 shell).

Higher level user oriented offerings include:

1. OpenLink Data Explorer front-end for exploring the burgeoning Linked Data Web
2. Ajax based SPARQL Query Builder (iSPARQL) that enables SPARQL Query construction by Example
3. Ajax based SQL Query Builder (QBE) that enables SQL Query construction by Example.

For Web 2.0 / 3.0 users, developers, and entrepreneurs it offers it includes Distributed Collaboration Tools & Social Media realm functionality courtesy of ODS that includes:

1. Point of presence on the Linked Data Web that meshes your Identity and your Data via URIs
2. System generated Social Network Profile & Contact Data via FOAF?
3. System generated SIOC (Semantically Interconnected Online Community) Data Space (that includes a Social Graph) exposing all your Web data in RDF Linked Data form
4. System generated OpenID and automatic integration with FOAF
5. Transparent Data Integration across Facebook, Digg, LinkedIn, FriendFeed, Twitter, and any other Web 2.0 data space equipped with RSS / Atom support and/or REST style Web Services
6. In-built support for SyncML which enables data synchronization with Mobile Phones.

How Do I Get Going with It?

• Standard Installation Guide
• Personal or Service Specific DBpedia Installation Guide

Typically, Orri's post are targeted at the hard core RDF and SQL DBMS audiences, but in this particular post, he shoots straight at the business community revealing "Opportunity Cost" containment as the invisible driver behind the business aspects of any market inflection.

Remember, the Web isn't ubiquitous because its users mastered the mechanics and virtues of HTML and/or HTTP. Web ubiquity is a function of the opportunity cost of not being on the Web, courtesy of the network effects of hyperlinked documents -- i.e., the instant gratification of traversing documents on the Web via a single click action. In similar fashion, the Linked Data Web's ubiquity will simply come down to the opportunity cost of not being "inside the Web", courtesy of the network effects of hyperlinked entities (documents, people, music, books, and other "Things").

Here are some excerpts from Orri's post:

Every time there is a major shift in technology, this shift needs to be motivated by addressing a new class of problem. This means doing something that could not be done before. The last time this happened was when the relational database became the dominant IT technology. At that time, the questions involved putting the enterprise in the database and building a cluster of line of business applications around the database. The argument for the RDBMS was that you did not have to constrain the set of queries that might later be made, when designing the database. In other words, it was making things more ad hoc. This was opposed then on grounds of being less efficient than the hierarchical and network databases which the relational eventually replaced. Today, the point of the Data Web is that you do not have to constrain what your data can join or integrate with, when you design your database. The counter-argument is that this is slow and geeky and not scalable. See the similarity? A difference is that we are not specifically aiming at replacing the RDBMS. In fact, if you know exactly what you will query and have a well defined workload, a relational representation optimized for the workload will give you about 10x the performance of the equivalent RDF warehouse. OLTP remains a relational-only domain. However, when we are talking about doing queries and analytics against the Web, or even against more than a handful of relational systems, the things which make RDBMS good become problematic.

If we think about Web 1.0 as a period where the distinguishing noun was: "Author", and Web 2.0 the noun: "Journalist", we should be able to see that what comes next is the noun: "Analyst". This new generation analyst would be equipped with de-referencable Web Identity courtesy of their Person Entity URI. The analyst's URI would also be the critical component of Web based low cost attribution ecosystem; one that ultimately turns the URI into the analyst's brand emblem / imprint.

Related

Paul Downey - Vanity of Demanding Attribution

You will control your data in the Web 3.0 realm. If somehow this remains somewhat incomprehensible and nebulous (as is typical in this emerging realm) then simply think about this as: The Magic of You!

Remember, "You" was the Times person of the year as an acknowledgement of the Web 2.0 phenomenon, and maybe this time next year it would simply be the "Magic of Being You" that's the person of the year :-)

Web 3.0 brings databasing to the Web (as a feature). The single most important action item at this stage is the act of creating a record for yourself, in this new distributed database held together by an HTTP based Network (e.g., the World Wide Web).

Related:

1. Get yourself a Web Database ID in 5 minutes or less
2. 2006 Callout from TimBL: Get Yourself a URI
3. Just watch the Numerati Video

From the RWW Top-Down category, which I interpret as: technologies that produce RDF from non RDF data sources. Our product portfolio is comprised of the following; Virtuoso Universal Server, OpenLink Data Spaces, OpenLink Ajax Toolkit, and OpenLink Data Explorer (which includes ubiquity commands).

Virtuoso Universal Server functionality summary:

1. Generation of RDF Linked Data Views of SQL, XML, and Web Services in general
2. Deployment of RDF Linked Data
3. "On the Fly" generation of RDF Linked Data from Document Web information resources (i.e. distillation of entities from their containers e.g. Web pages) via Cartridges / Drivers
4. SPARQL query language support
5. SPARQL extensions that bring SPARQL closer to SQL e.g Aggregates, Update, Insert, Delete Named Graph support (i.e. use of logical names to partition RDF data within Virtuoso's multi-model dbms engine)
6. Inference Engine (currently in use re. DBpedia via Yago and UMBEL)
7. Host and exposes data from Drupal, Wordpress, MediaWiki, phpBB3 as RDF Linked Data via in-built support for PHP runtime
8. Available as an EC2 AMI
9. etc..

OpenLink Data Spaces functionality summary:

1. Simple mechanism for Linked Data Web enabling yourself by giving you an HTTP based User ID (a de-referencable URI) that is linked to a FOAF based Profile page and OpenID
2. Binds all your data sources (blogs, wikis, bookmarks, photos, calendar items etc. ) to your URI so can "Find" things by only remembering your URI
3. Makes your profile page and personal URI the focal point of Linked Data Web presence
4. Delivers Data Portability (using data access by value or data access by reference) across data silos (e.g. Web 2.0 style social networks)
5. Allows you make annotations about anything in your own Data Space(s) on the Web without exposure to RDF markup
6. A Briefcase feature that provides a WebDAV driven RDF Linked Data variant of functionality seen in Mac OS X Spotlight and WinFS with the addition of SPARQL compliance
7. Automatically generates RDFa in its (X)HTML pages
8. Blog, Wiki, WebDAV File Server, Shared Bookmarks, Calendar, and other applications that look and feel like Web 2.0 counterparts but emitt RDF Linked Data amongst a plethora of data exchange formats
9. Available as an EC2 AMI
10. etc..

OpenLink Ajax Toolkit functionality summary:

1. Provides binding to SQL, RDF, XML, and Web Services via Ajax Database Connectivity Layer (you only need an ODBC, JDBC, OLE-DB, ADO.NET, XMLA Driver, or Web Service on the backend for dynamic data access from Javascript)
2. All controls are Ajax Database Connectivity bound (widgets get their data from Ajax Database Connectivity data sources)
3. Bundled with Virtuoso and ODS installations.
4. etc.

OpenLink Data Explorer functionality summary

1. Distills entities associated with information resource style containers (e.g. Web Pages or files) as RDF Linked Data
2. Exposes the RDF based Linked Data graph associated with information resources (see the Linked Data behind Web pages)
3. Ubiquity commands for invoking the above
4. Available as a Hosted Service or Firefox Extension
5. Bundled with Virtuoso and ODS installations
6. etc.

Note:

Of course you could have simply looked up OpenLink Software's FOAF based Profile page (*note the Linked Data Explorer tab*), or simply passed the FOAF profile page URL to a Linked Data aware client application such as: OpenLink Data Explorer, Zitgist Data Viewer, Marbles, and Tabulator, and obtained information. Remember, OpenLink Software is an Entity of Type: foaf:Organization, on the burgeoning Linked Data Web :-)

Related

• Linked Data Planet Keynote (RDFa based remix edition)
• On The Cusp: A Global Review of the Semantic Web Industry.

Understanding potential Linked Data Web business models, relative to other Web based market segments, is best pursued via a BCG Matrix diagram, such as the one I've constructed below:

Notes:

Link Density

• Web 1.0's collection of "Web Sites" have relatively low link density relative to Web 2.0's user-activity driven generation of semi-structured linked data spaces (e.g., Blogs, Wikis, Shared Bookmarks, RSS/Atom Feeds, Photo Galleries, Discussion Forums etc..)
• Semantic Technologies (i.e. "Semantics Inside style solutions") which are primarily about "Semantic Meaning" culled from Web 1.0 Pages also have limited linked density relative to Web 2.0
• The Linked Data Web, courtesy of the open-ended linking capacity of URIs, matches and ultimately exceeds Web 2.0 link density.

Relevance

• Web 1.0 and 2.0 are low relevance realms driven by hyperlinks to information resources ((X)HTML, RSS, Atom, OPML, XML, Images, Audio files etc.) associated with Literal Labels and Tagging schemes devoid of explicit property based resource description thereby making the pursuit of relevance mercurial at best
• Semantic Technologies offer more relevance than Web 1.0 and 2.0 based on the increased context that semantic analysis of Web pages accords
• The Linked Data Web, courtesy of URIs that expose self-describing data entities, match the relevance levels attained by Semantic Technologies.

Serendipity Quotient (SDQ)

• Web 1.0 has next to no serendipity, the closest thing is Google's "I'm Feeling Lucky" button
• Web 2.0 possess higher potential for serendipitous discovery than Web 1.0, but such potential is neutralized by inherent subjectivity due to its human-interaction-focused literal foundation (e.g., tags, voting schemes, wiki editors etc.)
• Semantic Technologies produce islands-of-relevance with little scope for serendipitous discovery due to URI invisibility, since the prime focus is delivering more context to Web search relative to traditional Web 1.0 search engines.
• The Linked Data Web's use of URIs as the naming and resolution mechanism for exposing structured and interlinked resources provides the highest potential for serendipitous discovery of relevant "Things"

To conclude, the Linked Data Web's market opportunities are all about the evolution of the Web into a powerful substrate that offers a unique intersection of "Link Density" and "Relevance", exploitable across horizontal and vertical market segments to solutions providers. Put differently, SDQ is how you take "The Ad" out of "Advertising" when matching Web users to relevant things :-)

A while back I watched Kevin Kelly's 5,000 days presentation at TED. During the presentation, I kept on scratching my head, wondering why phrases like "Linked Data", "Semantic Web", "Web of Data", "Data Web" where so unnaturally disconnected from his session narrative.

Yesterday I watched IMINDI's TechCrunch 50 presentation, and once again I saw the aforementioned pattern repeat itself. This time around, the poor founders of this "Linked Data Web" oriented company (which is what they are in reality) took a totally undeserved pasting from a bunch of panelist incapable of seeing beyond today (Web 2.0) and yesterday (initial Web bootstrap).

Anyway, thanks to the Web, this post will make a small contribution towards re-connecting the missing phrases to these "Linked Data Web" presentations.

Ubiquity from Mozilla Labs, provides an alternative entry point for experiencing the "Controller" aspect of the Web's natural compatibility with the MVC development pattern. As I've noted (in various posts) Web Services, as practiced by the REST oriented Web 2.0 community or SOAP oriented SOA community within the enterprise, is fundamentally about the ("Controller" aspect of MVC.

Ubiquity provides a commandline interface for direct invocation of Web Services. For instance, in our case, we can expose the Virtuoso's in-built RDF Middleware ("Sponger") and Linked Data deployment services via a single command of the form: describe-resource <url>

To experience this neat addition to Firefox you need to do the following:

1. Download and install the Ubiquity Extension for Firefox
2. Subscribe to the OpenLink Command for Resource Description
3. Click on CTRL+Space (Windows / Linux) or Option+Space (Mac OS X)
4. Type in: describe-resource <a-web-resource-url>

How to unsubscribe

1. To unsubscribe use type "about:ubiquity" into browser
2. Click on unsubscribe links associated with you command subscription list

Enjoy!

Jana: What are the benefits you see to the business community in adopting semantic technology?

Me: Exposure, exploitation, of untapped treasure trove of interlinked data, information, and knowledge across disparate IT infrastructure via conceptual entry points (Entity IDs / URIs / Data Source Names) that refer to as "Context Lenses".

Jana: Do you think these benefits are great enough for businesses to adopt the changes?

Me: Yes, infrastructural heterogeneity is a fact of corporate life (growth, mergers, acquisitions etc). Any technology that addresses these challenges is extremely important and valuable. Put differently, the opportunity costs associated with IT infrastructural heterogeneity remains high!

Jana: How large do you think this impact will actually be?

Me: Huge, enterprise have been aware of their data, information, and knowledge treasure troves etc. for eons. Tapping into these via a materialization of the "information at your fingertips" vision is something they've simply been waiting to pursue without any platform lock-in, for as long as I've been in this industry.

Jana: I’ve heard, from contacts in the Bay Area, that they are skeptical of how large this impact of semantic technology will actually be on the web itself, but that the best uses of the technology are for fields such as medical information, or as you mentioned, geo-spatial data.

Me: Unfortunately, those people aren't connecting the Semantic Web and open access to heterogeneous data sources, or the intrinsic value of holistic exploration location of entity based data networks (aka Linked Data).

Jana: Are semantic technologies going to be part of the web because of people championing the cause or because it is actually a necessary step?

Me: Linked Data technology on the Web is a vital extension of the current Web. Semantic Technology without the "Web" component, or what I refer to as "Semantics Inside only" solutions, simply offer little or no value as Web enhancements based on their incongruence with the essence of the Web i.e., "Open Linkage" and no Silos! A nice looking Silo is still a Silo.

Jana: In the early days of the web, there was an explosion of new websites, due to the ease of learning HTML, from a business to a person to some crackpot talking about aliens. Even today, CSS and XHTML are not so difficult to learn that a determined person can’t learn them from W3C or other tutorials easily. If OWL becomes the norm for websites, what do you think the effects will be on the web? Do you think it is easy enough to learn that it will be readily adopted as part of the standard toolkit for web developers for businesses?

Me: Correction, learning HTML had nothing to do with the Web's success. The value proposition of the Web simply reached critical mass and you simply couldn't afford to not be part of it. The easiest route to joining the Web juggernaut was a Web Page hosted on a Web Site. The question right now is: what's the equivalent driver for the Linked Data Web bearing in mind the initial Web bootstrap. My answer is simply this: Open Data Access i.e., getting beyond the data silos that have inadvertently emerged from Web 2.0.

Jana: Following the same theme, do you think this will lead to an internet full of corporate-controlled websites, with sites only written by developers rather than individuals?

Me: Not at all, we will have an Internet owned by it's participants i.e., You and the agents that work on your behalf.

Jana: So, you are imagining technologies such as Drupal or Wordpress, that allow users to manage sites without a great deal of knowledge of the nuts and bolts of current web technologies?

Me: Not at all! I envisage simple forms that provide conduits to powerful meshes of interlinked data spaces associated with Web users.

Jana: Given all of the buzz, and my own familiarity with ontology, I am just very curious if the semantic web is truly necessary?

Me:This question is no different than saying: I hear the Web is becoming a Database, and I wonder if a Data Dictionary is necessary, or even if access to structured data is necessary. It's also akin to saying: I accept "Search" as my only mechanism for Web interaction even though in reality, I really want to be able to "Find" and "Process" relevant things at a quicker rate than I do today, relative to the amount of information, and information processing time, at my disposal.

Jana: Will it be worth it to most people to go away from the web in its current form, with keyword searches on sites like Google, to a richer and more interconnected internet with potentially better search technology?

Me: As stated above, we need to add "Find" to the portfolio of functions we seek to perform against the Web. "Finding" and "Searching" are mutually inclusive pursuits at different ends of an activity spectrum.

Jana: For our more technical readers, I have a few additional questions: If no standardization comes about for mapping relational databases to domain ontologies, how do you see that as influencing the decisions about adoption of semantic technology by businesses? After all, the success of technology often lives or dies on its ease of adoption.

Me: Standardization of RDBMS to RDF Mapping is not the critical success factor here (of course it would be nice). As stated earlier, the issue of data integration that arises from IT infrastructural heterogeneity has been with decision makers in the enterprise for ever. The problem is now seeping into the broader consumer realm via Web ubiquity. The mistakes made in the enterprise realm are now playing out in the consumer Web realm. In both realms the critical success factors are:

1. Scalable productivity relative to exponential growth of data generated across Intranets, Extranets, and the Internet
2. Concept based Context Lenses that transcend logical and physical data heterogeneity by putting dereferencable URIs in front of the Line of Business Application Data and/or Web Data Spaces such as Blogs, Wikis, Discussion Forums etc.).

CrunchBase: When we released the CrunchBase API, you were one of the first developers to step up and quickly released a CrunchBase Sponger Cartridge. Can you explain what a CrunchBase Sponger Cartridge is?

Me: A Sponger Cartridge is a data access driver for Web Resources that plugs into our Virtuoso Universal Server (DBMS and Linked Data Web Server combo amongst other things). It uses the internal structure of a resource and/or a web service associated with a resource, to materialize an RDF based Linked Data graph that essentially describes the resource via its properties (Attributes & Relationships).

CrunchBase: And what inspired you to create it?

Me: Bengee built a new space with your data, and we've built a space on the fly from your data which still resides in your domain. Either solution extols the virtues of Linked Data i.e. the ability to explore relationships across data items with high degrees of serendipity (also colloquially known as: following-your-nose pattern in Semantic Web circles).

Bengee posted a notice to the Linking Open Data Community's public mailing list announcing his effort. Bearing in mind the fact that we've been using middleware to mesh the realms of Web 2.0 and the Linked Data Web for a while, it was a no-brainer to knock something up based on the conceptual similarities between Wikicompany and CrunchBase. In a sense, a quadrant of orthogonality is what immediately came to mind re. Wikicompany, CrunchBase, Bengee's RDFization efforts, and ours.

Bengee created an RDF based Linked Data warehouse based on the data exposed by your API, which is exposed via the Semantic CrunchBase data space. In our case we've taken the "RDFization on the fly" approach which produces a transient Linked Data View of the CrunchBase data exposed by your APIs. Our approach is in line with our world view: all resources on the Web are data sources, and the Linked Data Web is about incorporating HTTP into the naming scheme of these data sources so that the conventional URL based hyperlinking mechanism can be used to access a structured description of a resource, which is then transmitted using a range negotiable representation formats. In addition, based on the fact that we house and publish a lot of Linked Data on the Web (e.g. DBpedia, PingTheSemanticWeb, and others), we've also automatically meshed Crunchbase data with related data in DBpedia and Wikicompany data.

CrunchBase: Do you know of any apps that are using CrunchBase Cartridge to enhance their functionality?

Me: Yes, the OpenLink Data Explorer which provides CrunchBase site visitors with the option to explore the Linked Data in the CrunchBase data space. It also allows them to "Mesh" (rather than "Mash") CrunchBase data with other Linked Data sources on the Web without writing a single line of code.

CrunchBase: You have been immersed in the Semantic Web movement for a while now. How did you first get interested in the Semantic Web?

Me: We saw the Semantic Web as a vehicle for standardizing conceptual views of heterogeneous data sources via context lenses (URIs). In 1998 as part of our strategy to expand our business beyond the development and deployment of ODBC, JDBC, and OLE-DB data providers, we decided to build a Virtual Database Engine (see: Virtuoso History), and in doing so we sought a standards based mechanism for the conceptual output of the data virtualization effort. As of the time of the seminal unveiling of the Semantic Web in 1998 we were clear about two things, in relation to the effects of the Web and Internet data management infrastructure inflections: 1) Existing DBMS technology had reached it limits 2) Web Servers would ultimately hit their functional limits. These fundamental realities compelled us to develop Virtuoso with an eye to leveraging the Semantic Web as a vehicle from completing its technical roadmap.

CrunchBase: Can you put into layman’s terms exactly what RDF and SPARQL are and why they are important? Do they only matter for developers or will they extend past developers at some point and be used by website visitors as well?

Me: RDF (Resource Description Framework) is a Graph based Data Model that facilitates resource description using the Subject, Predicate, and Object principle. Associated with the core data model, as part of the overall framework, are a number of markup languages for expressing your descriptions (just as you express presentation markup semantics in HTML or document structure semantics in XML) that include: RDFa (simple extension of HTML markup for embedding descriptions of things in a page), N3 (a human friendly markup for describing resources), RDF/XML (a machine friendly markup for describing resources).

SPARQL is the query language associated with the RDF Data Model, just as SQL is a query language associated with the Relational Database Model. Thus, when you have RDF based structured and linked data on the Web, you can query against Web using SPARQL just as you would against an Oracle/SQL Server/DB2/Informix/Ingres/MySQL/etc.. DBMS using SQL. That's it in a nutshell.

CrunchBase: On your website you wrote that “RDF and SPARQL as productivity boosters in everyday web development”. Can you elaborate on why you believe that to be true?

Me: I think the ability to discern a formal description of anything via its discrete properties is of immense value re. productivity, especially when the capability in question results in a graph of Linked Data that isn't confined to a specific host operating system, database engine, application or service, programming language, or development framework. RDF Linked Data is about infrastructure for the true materialization of the "Information at Your Fingertips" vision of yore. Even though it's taken the emergence of RDF Linked Data to make the aforementioned vision tractable, the comprehension of the vision's intrinsic value have been clear for a very long time. Most organizations and/or individuals are quite familiar with the adage: Knowledge is Power, well there isn't any knowledge without accessible Information, and there isn't any accessible Information without accessible Data. The Web has always be grounded in accessibility to data (albeit via compound container documents called Web Pages).

Bottom line, RDF based Linked Data is about Open Data access by reference using URIs (HTTP based Entity IDs / Data Object IDs / Data Source Names), and as I said earlier, the intrinsic value is pretty obvious bearing in mind the costs associated with integrating disparate and heterogeneous data sources -- across intranets, extranets, and the Internet.

CrunchBase: In his definition of Web 3.0, Nova Spivack proposes that the Semantic Web, or Semantic Web technologies, will be force behind much of the innovation that will occur during Web 3.0. Do you agree with Nova Spivack? What role, if any, do you feel the Semantic Web will play in Web 3.0?

Me: I agree with Nova. But I see Web 3.0 as a phase within the Semantic Web innovation continuum. Web 3.0 exists because Web 2.0 exists. Both of these Web versions express usage and technology focus patterns. Web 2.0 is about the use of Open Source technologies to fashion Web Services that are ultimately used to drive proprietary Software as Service (SaaS) style solutions. Web 3.0 is about the use of "Smart Data Access" to fashion a new generation of Linked Data aware Web Services and solutions that exploit the federated nature of the Web to maximum effect; proprietary branding will simply be conveyed via quality of data (cleanliness, context fidelity, and comprehension of privacy) exposed by URIs.

Here are some examples of the CrunchBase Linked Data Space, as projected via our CruncBase Sponger Cartridge:

1. Amazon.com
2. Microsoft
3. Google
4. Apple

My contribution to the developing discourse takes the form of a Q&A session. I've taken the questions posed and provided answers that express my particular points of view:

Q: Is the desktop of the future going to just be a web-hosted version of the same old-fashioned desktop metaphors we have today?

A: No, it's going to be a more Web Architecture aware and compliant variant exposed by appropriate metaphors.

Q: The desktop of the future is going to be a hosted web service

A: A vessel for exploiting the virtues of the Linked Data Web.

Q: The Browser is Going to Swallow Up the Desktop

A: Literally, of course not! Metaphorically, of course! And then the Browser metaphor will decomposes into function specific bits of Web interaction amenable to orchestration by its users.

Q: The focus of the desktop will shift from information to attention

A: No! Knowledge, Information, and Data sharing courtesy of Hyperdata & Hypertext Linking.

Q: Users are going to shift from acting as librarians to acting as daytraders

A: They were Librarians at Web 1.0, Journalist at Web 2.0, and Analysts in Web 3.0 (i.e, analyze structured and interlinked data), and CEOs in Web 4.0 (i.e. get Agents to do stuff intelligently en route to making decisions).

Q: The Webtop will be more social and will leverage and integrate collective intelligence

A: The Linked Data Web vessel will only require you to fill in your profile (once) and then serendipitous discovery and meshing of relevant data will simply happen (the serendipity quotient will grow in line with Linked Data Web density).

Q: The desktop of the future is going to have powerful semantic search and social search capabilities built-in

A: It is going to be able to "Find" rather than "Search" for stuff courtesy of the Linked Data Web.

Q: Interactive shared spaces will replace folders

A: Data Spaces and their URIs (Data Source Names) replace everything. You simply choose the exploration metaphor that best suits you space interaction needs.

Q: The Portable Desktop

A: Ubiquitous Desktop i.e. do the same thing (all answers above) on any device connected to the Web.

Q: The Smart Desktop

A: Vessels with access to Smart Data (Linked Data + Action driven Context sprinklings).

Q: Federated, open policies and permissions

A: More federation for sure, XMPP will become a lot more important, and OAuth will enable resurgence of the federated aspects of the Web and Internet.

Q: The personal cloud

A: Personal Data Spaces plugged into Clouds (Intranet, Extranet, Internet).

Q: The WebOS

A: An operating system endowed with traditional Database and Host Operating system functionality such as: RDF Data Model, SPARQL Query Language, URI based Pointer mechanism, and HTTP based message Bus.

Q: Who is most likely to own the future desktop?

A: You! And all you need is a URI (an ID or Data Source Name for "Entity You") and a Profile Page (a place where "Entity You" is Describe by You).

One Last Thing

You can get a feel for the future desktop by downloading and then installing the OpenLink Data Explorer plugin for Firefox, which allows you to switch viewing modes between Web Page and Linked Data behind the page. :-)

Related

• OpenLink Data Spaces
• Get Yourself a URI in 5 Minutes or Less
• Linked Data Spaces & Data Portability
• Linked Data Conference Keynote (RDFa based remix edition that includes vital bits from TimBL's Linked Data Planet presentation).

How Semantic Tagging Works (from a 1000 feet)

When tagging a document, the semantic tagging service passes the content of a target document through a processing pipeline (a distillation process of sorts) that results in automagic extraction of the following:

-- Named Entities

-- Subject matter Entities (Subject matter Concepts reflecting topics covered by the document

Once the extraction phase is completed, a user is presented with a list of "suggested tags" using a variety of user interaction techniques. The literal values of elected Tags are then associated with one or more Tag and Tag Meaning Data Objects, with each Object type endowed with a unique Identifier.

Issues to Note

Broad acceptance that: "Context is king", is gradually taking shape. That said, "Context" landlocked within Literal values offers little over what we have right now (e.g. at Del.icio.us or Technorati), long term. By this I mean: if the end product of semantically enhanced tagging leaves us with: Literal Tag values only, Tags associated with Tag Data Objects endowed with platform specific Identifiers, or Tag Data Objects with any other Identity scheme that excludes HTTP, the ability of Web users to discern or derive multiple perspectives from the base Context (exposed by semantically enhanced Tags) will be lost, or severely impeded at best.

The shape, form, and quality of the lookup substrate that underlies semantic tagging services, ultimately affects "context fidelity" matters such as Entity Disambiguation. The importance of quality lookup infrastructure on the burgeoning Linked Data Web is the reason why OpenLink Software is intimately involved with the DBpedia and UMBEL projects.

Conclusions

I am immensely happy to see that the Web 2.0 and Semantic Web communities are beginning to coalesce around the issue of "Context". This was the case at the WWW2008 Linked Data Workshop, I am feeling a similar vibe emerging from the Semantic Web Technologies conference currently nearing completion in San Jose. Of course, I will be talking about, and demonstrating practical utility of all of this, at the upcoming Linked Data Planet conference.

Related

My Data Space Tag Cloud (*a Linked Data Space*)

Faviki (note: this service needs to expose Linked Data compliant Tag URIs)

MOAT Ontology

ODBC identifies data sources using Data Source Names (DSNs).

WODBC (Web Open Database Connectivity) delivers open data access to Web Databases / Data Spaces. The Data Source Naming scheme: URI or IRI, is HTTP based thereby enabling data access by reference via the Web.

ODBC DSNs bind ODBC client applications to Tables, Views, Stored Procedures.

WODBC DSNs bind you to a Data Space (e.g. my FOAF based Profile Page where you can use the "Explore Data Tab" to look around if you are a human visitor) or a specific Entity within a Data Space (i.e Person Entity Me).

ODBC Drivers are built using APIs (DBMS Call Level Interfaces) provided by DBMS vendors. Thus, a DBMS vendor can chose not to release an API, or do so selectivity, for competitive advantage or market disruption purposes (it's happened!).

WODBC Drivers are also built using APIs (Web Services associated with a Web Data Space). These drivers are also referred to as RDF Middleware or RDFizers. The "Web" component of WODBC ensures openness, you publish Data with URIs from your Linked Data Server and that's it; your data space or specific data entities are live and accessible (by reference) over the Web!

So we have come full circle (or cycle), the Web is becoming more of a structured database everyday! What's new is old, and what's old is new!

Data Access is everything, without "Data" there is no information or knowledge. Without "Data" there's not notion of vitality, purpose, or value.

URIs make or break everything in the Linked Data Web just as ODBC DSNs do within the enterprise.

I've deliberately left JDBC, ADO.NET, and OLE-DB out of this piece due to their respective programming languages and frameworks specificity. None of these mechanisms match the platform availability breadth of ODBC.

The Web as a true M-V-C pattern is now crystalizing. The "M" (Model) component of M-V-C is finally rising to the realm of broad attention courtesy of the "Linked Data" meme and "Semantic Web" vision.

By the way, M-V-C lines up nicely with Web 1.0 (Web Forms / Pages), Web 2.0 (Web Services based APIs), and Web 3.0 (Data Web, Web of Data, or Linked Data Web) :-)

Evoluation is evolution devoid of the randomness of mutation. A state of being in which it is possible to evaluate and choose evolutionary paths.

Evoluation actually describes where we are today in relation to the World Wide Web; to the Linking Open Data community (LOD), it's taking the path towards becoming a Giant Global Graph of Linked Data; to the Web 2.0 community, it's simply a collection of Web Services and associated APIs; and to many others, it remains an opaque collection of interlinked documents.

The great thing about the Web is that it allows netizens to explore a plethora of paths without adversely affecting the paths of others. That said, controlling one's path may take mutation out of evolution, but we are still left with the requirement to adapt and eventually survive in a competitive environment. Thus, although we can evaluate and choose from the many paths the Web's evolution offers us, the path that delivers the most benefits ultimately dominates. :-)

Daniel simplifies my post by using diagrams to depict the different paths for PHP based applications exposing Linked Data - especially those that already provide a significant amount of the content that drives Web 2.0.

If all the content in Web 2.0 information resources are distillable into discrete data objects endowed with HTTP based IDs (URIs), with zero "RDF handcrafting Tax", what do we end up with? A Giant Global Graph of Linked Data; the Web as a Database.

So, what used to apply exclusively, within enterprise settings re. Oracle, DB2, Informix, Ingres, Sybase, Microsoft SQL Server, MySQL, PostrgeSQL, Progress Open Edge, Firebird, and others, now applies to the Web. The Web becomes the "Distributed Database Bus" that connects database records across disparate databases (or Data Spaces). These databases manage and expose records that are remotely accessible "by reference" via HTTP.

As I've stated at every opportunity in the past, Web 2.0 is the greatest thing that every happened to the Semantic Web vision :-) Without the "Web 2.0 Data Silo Conundrum" we wouldn't have the cry for "Data Portability" that brings a lot of clarity to some fundamental Web 2.0 limitations that end-users ultimately find unacceptable.

In the late '80s, the SQL Access Group (now part of X/Open) addressed a similar problem with RDBMS silos within the enterprise that lead to the SAG CLI which is exists today as Open Database Connectivity.

In a sense we now have WODBC (Web Open Database Connectivity), comprised of Web Services based CLIs and/or traditional back-end DBMS CLIs (ODBC, JDBC, ADO.NET, OLE-DB, or Native), Query Language (SPARQL Query Language), and a Wire Protocol (HTTP based SPARQL Protocol) delivering Web infrastructure equivalents of SQL and RDA, but much better, and with much broader scope for delivering profound value due to the Web's inherent openness. Today's PHP, Python, Ruby, Tcl, Perl, ASP.NET developer is the enterprise 4GL developer of yore, without enterprise confinement. We could even be talking about 5GL development once the Linked Data interaction is meshed with dynamic languages (delivering higher levels of abstraction at the language and data interaction levels). Even the underlying schemas and basic design will evolve from Closed World (solely) to a mesh of Closed & Open World view schemas.

Yes, integration is hard, but I do profoundly believe that what's been happening on the Web over the last 10 or so years also applies to the Enterprise, and by this I absolutely do not mean "Enterprise 2.0" since "2.0" and productive agility do not compute in my realm of discourse.

large collections of RSS feeds, Wikiwords, Shared Bookmarks, Discussion Forums etc.. when disconnected at the data level (i.e. hosted in pages with no access to the "data behind") simply offer information deluge and inertia (there are only so many hours for processing opaque information sources in a given day).

Enterprises fundamentally need to process information efficiently as part of a perpetual assessment of their relative competitive Strengths, Weaknesses, Opportunities, and Threats (SWOT), in existing and/or future markets. Historically, IT acquisitions have run counter intuitively to the aforementioned quest for "Ability" due to the predominance of "rip and replace" approach technology acquisition that repeatedly creates and perpetuates information silos across Application, Database, Operating System, Development Environment boundaries. The sequence of events typically occurs as follows:

1. applications are acquired on a problem by problem basis
2. back-end application databases are discovered once ad-hoc information views are sought by information workers
3. back-end database disparity across applications is discovered once holistic views are sought by knowledge workers (typically domain experts).

In the early to mid 90's (pre ubiquitous Web), operating system, programming language, operating system, and development framework independence inside the enterprise was technically achievable via ODBC (due to it's platform independence). That said, DBMS specific ODBC channels alone couldn't address the holistic requirements associated with Conceptual Views of disparate data sources, hence the need for Data Access Virtualization via Virtual Database Engine technology.

Just as is the case on the Web today, with the emergence of the "Linked Data" meme, enterprises now have a powerful mechanism for exploiting the Data Integration benefits associated with generating Data Objects from disparate data sources, endowed with HTTP based IDs (URIs).

Conceptualizing access to data exposed Databases APIs, SOA based Web Services (SOAP style Web Services), Web 2.0 APIs (REST style Web Services), XML Views of SQL Data (SQLX), pure XML etc.. is problem area addressed by RDF aware middleware (RDFizers e.g Virtuoso Sponger).

Here are examples of what SQL Rows exposed as RDF Data Objects (identified using HTTP based URIs) would look like outside or behind a corporate firewall:

Customer - Alfreds Futterkiste

Customer Contact - Maria Anders

Salesrep - Nancy Davolio

Customer Orders Numbers - 11084, 11011, 11078, 11085

What's Good for the Web Goose (Personal Data Space URIs) is good for the Enterprise Gander (Enterprise Data Space URIs).

Related

Data Access - A Cultural or Technical Challenge?

BTW - Benjamin Nowack penned an interesting post titled: Semantic Web Aliases, that covers a variety of labels used to describe the Semantic Web. The great thing about this post is that it provides yet another demonstration-in-the-making for the virtues of Linked Data :-)

Labels are harmless when their sole purpose is the creation of routes of comprehension for concepts. Unfortunately, Labels aren't always constructed with concept comprehension in mind, most of the time they are artificial inflectors and deflectors servicing marketing communications goals.

Anyway, irrespective of actual intent, I've endowed all of the labels from Bengee's post with URIs as my contribution important disambiguation effort re. the Semantic Web:

• Semantic Web (timbl)
• Web of Data (timbl)
• lowercase semantic [wW]eb (tantek)
• Semantic Web 2.0 (by stefandecker, IIRC)
• Web 3.0 (by nova and others)
• Semantic Graph (by nova and others)
• Hyperdata (by danja) Linked Data (by timbl, and implemented by the Chris Bizer and Richard Cyganiak inspired, Linking Open Data Community and it's poster project DBpedia)
• Linked Data Web (by kidehen)
• Structured Web (by mkbergman)
• Semantic Data Web (by kidehen)
• SemWeb (by the developer community)
• GGG - The Giant Global Graph (by timbl) Web 3G (by iand)

As per usual this post is best appreciated when processed via an Linked Data aware user agent.

OpenLink Data Spaces (ODS) now officially supports:

• Attention Profiling Markup Language (APML).
• Meaning of a Tag (MOAT) in conjunction with Simple Knowledge Organisation System (SKOS) and Social-Semantic Cloud of Tags (SCOT).
• OAuth - an Open Authentication Protocol

Which means that OpenLink Data Spaces support all of the main standards being discussed in the DataPortability Interest Group!

APML Example:

All users of ODS automatically get a dynamically created APML file, for example: APML profile for Kingsley Idehen

The URI for an APML profile is: http://myopenlink.net/dataspace/<ods-username>/apml.xml

Meaning of a Tag Example:

All users of ODS automatically have tag cloud information embedded inside their SIOC file, for example: SIOC for Kingsley Idehen on the Myopenlink.net installation of ODS.

But even better, MOAT has been implemented in the ODS Tagging System. This has been demonstrated in a recent test blog post by my colleague Mitko Iliev, the blog post comes up on the tag search: http://myopenlink.net/dataspace/imitko/weblog/Mitko%27s%20Weblog/tag/paris

Which can be put through the OpenLink Data Browser:

• OpenLink Data Browser with Mitko Iliev’s Paris Blog Tag

OAuth Example:

OAuth Tokens and Secrets can be created for any ODS application. To do this:

1. you can log in to MyOpenlink.net beta service, the Live Demo ODS installation, an EC2 instance, or your local installation
2. then go to ‘Settings’
3. and then you will see ‘OAuth Keys’
4. you will then be able to choose the applications that you have instantiated and generate the token and secret for that app.

Related Document (Human) Links

• OpenLink Data Spaces Official Page
• OpenLink Software Page
• OpenLink Data Spaces Wikipedia Page
• Attention Profiling Markup Language Project Website
• Meaning of a Tag Project Website
• Simple Knowledge Organisation Systems Project Website
• Social-Semantic Cloud of Tags Project Website
• OAuth Protocol Website
• DataPortability.org Website
• Semantically Interlinked Online Communities Project Website
  

Remember (as per my most recent post about ODS), ODS is about unobtrusive fusion of Web 1.0, 2.0, and 3.0+ usage and interaction patterns. Thanks to a lot of recent standardization in the Semantic Web realm (e.g SPARQL), we are now employ the MOAT, SKOS, and SCOT ontologies as vehicles for Structured Tagging.

Structured Tagging?

This is how we take a key Web 2.0 feature (think 2D in a sense), bend it over, to create a Linked Data Web (Web 3.0) experience unobtrusively (see earlier posts re. Dimensions of Web). Thus, nobody has to change how they tag or where they tag, just expose ODS to the URLs of your Web 2.0 tagged content and it will produce URIs (Structured Data Object Identifiers) and a lnked data graph for your Tags Data Space (nee. Tag Cloud). ODS will construct a graph which exposes tag subject association, tag concept alignment / intended meaning, and tag frequencies, that ultimately deliver "relative disambiguation" of intended Tag Meaning (i.e. you can easily discern the taggers meaning via the Tags actual Data Space which is associated with the tagger). In a nutshell, the dynamics of relevance matching, ranking, and the like, change immensely without futile timeless debates about matters such as:

What's the Linked Data value proposition?

What's the Linked Data business model?

XML vs RDF

XQuery vs SPARQL

What's the Semantic Web Killer application?

We can just get on with demonstrating Linked Data value using what exists on the Web today. This is the approach we are deliberately taking with ODS.

Related Items

Stefano Mazzocch's response to Clay Shirky's 2005 talk titled: Ontology is Overrated: Links, Tags and Post-hoc Metadata

Tom Gruber's post titled: Ontology of Folksonomy: A Mash-up of Apples and Oranges

Tip: This post is best viewed via an RDF aware User Agent (e.g. a Browser or Data Viewer). I say this because the permalink of this post is a URI in a Linked Data Space (My Blog) comprised of more data than meets the eye (i.e. what you see when you read this post via a Document Web Browser) :-)

There are quite a few reasons to use OpenLink Data Spaces (ODS). Here are 10 of the reasons why I use ODS:

1. Its native support of DataPortability Recommendations such as RSS, Atom, APML, Yadis, OPML, Microformats, FOAF, SIOC, OpenID and OAuth.
2. Its native support of Semantic Web Technologies such as: RDF and SPARQL/SPARUL for querying.
3. Everything in ODS is an Object with its own URI, this is due to the underlying Object-Relational Architecture provided by Virtuoso.
4. It has all the social media components that you could need, including: blogs, wikis, social networks, feed readers, CRM and a calendar.
5. It is expandable by installing pre-configured components (called VADs), or by re-configuring a LAMP application to use Virtuoso. Some examples of current VADs include: MediaWiki, Wordpress and Drupal.
6. It works with external webservices such as: Facebook, del.icio.us and Flickr.
7. Everything within OpenLink Data Spaces is Linked Data, which provides more meaningful information than just plain structural information. This meaningful information could be used for complex inferencing systems, as ODS can be seen as a Knowledge Base.
8. ODS builds bridges between the existing static-document based web (aka ‘Web 1.0‘), the more dynamic,  services-oriented, social and/or user-orientated webs (aka ‘Web 2.0‘) and the web which we are just going into, which is more data-orientated (aka ‘Web 3.0’ or ‘Linked Data Web’).
9. It is fully supportive of Cloud Computing, and can be installed on Amazon EC2.
10. Its released free under the GNU General Public License (GPL). [note]However, it is technically dual licensed as it lays on top of the Virtuoso Universal Server which has both Commercial and GPL licensing[/note]

The features above collectively provide users with a Linked Data Junction Box that may reside with corporate intranets or "out in the clouds" (Internet). You can consume, share, and publish data in a myriad of formats using a plethora of protocols, without any programming. ODS is simply about exposing the data from your Web 1.0, 2.0, 3.0 application interactions in structured from, with Linking, Sharing, and ultimately Meshing (not Mashing) in mind.

Note: Although ODS is equipped with a broad array of Web 2.0 style Applications, you do not need to use native ODS apps in order to exploit it's power. It binds to anything that supports the relevant protocols and data formats.

Jason recently moved to Massachusetts which lead to me pinging him about our earlier blogosphere encounter and the emergence of a Data Portability Community. I also informed him about the fact that TimBL, myself, and a number of other Semantic Web technology enthusiasts, frequently meet on the 2nd Tuesday of each month at the MIT hosted Cambridge Semantic Web Gatherings, to discuss, demonstrate, debate all aspects of the Semantic Web. Luckily (for both of us), Jason attended the last event, and we got to meet each other in person.

Following our face to face meeting in Cambridge, a number of follow-on conversations ensued covering, Linked Data and practical applications of the Semantic Web vision. Jason writes about our exchanges a recent post titled: The Semantic Web. His passion for Data Portability enabled me to use OpenID and FOAF integration to connect the Semantic Web and Data Portability via the Linked Data concept.

During our conversations, Jason also eluded to the fact that he had already encountered OpenLink Software while working with our ODBC Drivers (part of or UDA product family) for IBM Informix (Single-Tier or Multi-Tier Editions) a few years ago (interesting random connection).

As I've stated in the past, I've always felt that the Semantic Web vision will materialize by way of a global epiphany. The count down to this inevitable event started at the birth of the blogosphere, ironically. And accelerated more recently, through the emergence of Web 2.0 and Social Networking, even more ironically :-)

The blogosphere started the process of Data Space coalescence via RSS/Atom based semi-strucutured data enclaves, Web 2.0 RDFpropagated Web Service usage en route to creating service provider controlled, data and information silosRDF, Social NetworkingRDF brought attention to the fact that User Generated Data wasn't actually owned or controlled by the Data Creators etc.

The emergence of "Data Portability" has created a palatable moniker for a clearly defined, and slightly easier to understand, problem: the meshing of Data and Identity in cyberspace i.e. individual points of presence in cyberspace, in the form of "Personal Data Spaces in the Clouds" (think: doing really powerful stuff with .name domains). In a sense, this is the critical inflection point between the document centric "Web of Linked Documents" and the data centric "Web or Linked Data". There is absolutely no other way solve this problem in a manner that alleviates the imminent challenges presented by information overload -- resulting from the exponential growth of user generated data across the Internet and enterprise Intranets.

1. Web 2.0 is fundamentally about Web Services as the prime vehicle for interactions across "points of Web presence"
2. Facebook is a Web 2.0 hosted service for social networking that provides Web Services APIs for accessing data in the Facebook data space. You have to do so "on the fly" within clearly defined constraints i.e you can interact with data across your social network via Facebook APIs, but you cannot cache the data (perform an export style dump of the data)
3. Facebook is a main driver of the term: "social graph", but their underlying data model is relational and the Web Services response (data you get back) doesn't return a data graph, instead it returns an tree (i.e XML)
4. Scoble's had a number of close encounters with Linked Data Web | Semantic Data Web | Web 3.0 aficionados in various forms throughout 2007, but still doesn't quite make the connection between Web Services APIs as part of a processing pipeline that includes structured data extraction from XML data en route to producing Data Graphs comprised of Data Objects (Entities) endowed with: Unique Identifiers, Classification or Categorization schemes, Attributes, and Relationships prescribed by one or more shared Data Dictionaries/Schemas/Ontologies
5. A global information bus that exposes a Linked Data mesh comprised of Data Objects, Object Attributes, and Object Relationships across "points of Web presence" is what TimBL described in 1998 (Semantic Web Roadmap) and more recently in 2007 (Giant Global Graph)
6. The Linked Data mesh (i.e Linked Data Web or GGG) is anchored by the use of HTTP to mint Location, Structure, and Value independent Object Identifiers called URIs or IRIs. In addition, the Linked Data Web is also equipped with a query language, protocol, and results serialization format for XML and JSON called: SPARQL.

So, unlike Scoble, I am able to make my Facebook Data portable without violating Facebook rules (no data caching outside Facebook realm) by doing the following:

1. Use an RDFizer for Facebook to convert XML response data from Facebook Web Services into RDF "on the fly" Ensure that my RDF is comprised of Object Identifiers that are HTTP based and thereby dereferencable (i.e. I can use SPARQL to unravel the Linked Data Graph in my Facebook data space)
2. The act of data dereferencing enables me to expose my Facebook Data as Linked Data associated with my Personal URI
3. This interaction only occurs via my data space and in all cases the interactions with data work via my RDFizer middleware (e.g the Virtuoso Sponger) that talks directly to Facebook Web Services.

In a nutshell, my Linked Data Space enables you to reference data in my data space via Object Identifiers (URIs), and some cases the Object IDs and Graphs are constructed on the fly via RDFization middleware.

Here are my URIs that provide different paths to my Facebook Data Space:

Personal URI

My Facebook Data Space (best viewed via a Linked Data Browser/Viewer session)

My Facebook Photo Gallery -- WWW2007 Photo Collection (also best viewed via a Linked Data Browser/Viewer session)

To conclude, 2008 is clearly the inflection year during which we will final unshackle Data and Identity from the confines of "Web Data Silos" by leveraging the HTTP, SPARQL, and RDF induced virtues of Linked Data.

Related Posts:

1. 2008 and the Rise of Linked Data
2. Scoble Right, Wrong, and Beyond
3. Scoble interviewing TimBL (note to Scoble: re-watch your interview since he made some specific points about Linked Data and URIs that you need to grasp)
4. Prior Blog posts my this Blog Data Space that include the literal patterns: Scoble Semantic Web

* 64-bit Integer Support

* RDF Sink Folders for WebDAV - enabling RDF Quad Store population by simply dropping RDF files into WebDAV or via HTTP (meaning you can use CURL as an RDF in put mechanism for instance)

* Additional Sponger Cartridges from Audio binary files (i.e ID3 tag extraction and Music Ontology mapping which exposes the fine details of music as RDF based Structured Data; one for the DJs & Remixers out there!)

* New Sponger Cartridges for Facebook, Freebase, Wikipedia, GRDDL, RDFa, eRDF and more

* Support for PHP 5.2 runtime hosting (Virtuoso is a bona fide deployment platform for: Wordpress, MediaWiki, phpBB, Drupal etc.)

* Enhanced UI for managing RDF Linked Data deployment (covering Multi Homed domains, Virtual Directories associated with URL-rewrite rules

* Demonstration Database includes SQL-RDF Views & SQL Table samples for the THALIA Web Data Integration benchmark and test-suite

* Tutorial Application includes Linked Data style SQL-RDF Views for the Northwind SQL DBMS schema (which is the same as the standard Virtuoso demo atabase schema)

* SQL-RDF Views implementation of the TPC-D benchmark (Yes, we can run this grueling SQL benchmark via RDF views of SQL Data!)

* A new Amazon EC2 Image for Virtuoso that enables you to instantiate a fully configured instance comprising the Virtuoso core, OpenLink Data Spaces platform and the OpenLink Ajax Toolkit (OAT) (we now have bona fide Data Spaces in the Clouds as an addition to the emerging Semantic Data Web mesh).

Download Lnks:

* Open Source Edition

* Commercial Edition

Jon Udell recently penned a post titled: The Fourth Platform. The post arrives at a spookily coincidental time (this happens quite often between Jon and I as demonstrated last year during our podcast; the "Fourth" in his Innovators Podcast series).

The platform that Jon describes is "Cloud Based" and comprised of Storage and Computation. I would like to add Data Access and Management (native and virtual) under the fourth platform banner with the end product called: "Cloud based Data Spaces".

As I write, we are releasing a Virtuoso AMI (Amazon Image) labeled: virtuoso-dataspace-server. This edition of Virtuoso includes the OpenLink Data Spaces Layer and all of the OAT applications we've been developing for a while.

What Benefits Does this offer?

1. Personal Data Spaces in the Cloud - a place where you can control and consolidate data across your Blogs, Wikis, RSS/Atom Feed Subscriptions, Shared Bookmarks, Shared Calendars, Discussion Threads, Photo Galleries etc
2. All the data in your Data Space is SPARQL or GData accessible.
3. All of the data in your Personal Data Space is Linked Data from the get go. Each Item of data is URI addressable
4. SIOC support - your Blogs, Wikis, Bookmarks etc.. are based on the SIOC ontology for Semantically Interlinking Online Communities (think: Open social-graph++)
5. FOAF support - your FOAF Profile page provides a URI that is an in-road to all Data in your Data Space.
6. OpenID support - your Personal Data Space ID is usable wherever OpenID is supported. OpenID and FOAF are integrated as per latest FOAF specs
7. Two Integration with Facebook - You can access your Data Space from Facebook or access Facebook from your Data Space
8. Unified Storage - The WebDAV based filesystem provides Cloud Storage that's integrated with Amazon S3; It also exposes all of your Data Space data via a traditional filesystem UI (think virtual Spotlight); You can also mount this drive to your local filesystem via your native operating system's WebDAV support
9. SyncML - you can sync calendar and contact details with your Data Space in the cloud from your Mobile phone.
10. A practical Semantic Data Web solution - based on Web Infrastructure and doesn't require you to do anything beyond exposing URIs for data in your Data Spaces.

EC2-AMI Details:

AMI ID: ami-e2ca2f8b

Manifest file: virtuoso-images/virtuoso-dataspace-server.manifest.xml

Installation Guide:

1. Get an Amazon Web Services (AWS) account
2. Signup for S3 and EC2 services
3. Install the EC2 plugin for Firefox
4. Start the EC2 plugin
5. Locate the row containing ami-7c31d515  Manifest virtuoso-test/virtuoso-cloud-beta-9-i386.manifest.xml (sort using the AMI ID or Manifest Columns or search on pattern: virtuoso, due to name flux)
6. Start the Virtuoso Data Space Server AMI
7. Wait 4-5 minutes (*take a few minutes to create the pre-configured Linux Image*)
8. Connect to http://http://your-ec2-instance-cname:8890/ Log in with user/password dba/dba
9. Go to the Admin UI (Virtuoso Conductor) and change the PWDs for the 'dba' and 'dav' accounts (*Important!*)
10. Give the "SPARQL" user "SPARQL_UPDATE" privileges (required if you want to exploit the in-built Sponger Middleware)
11. Click on the ODS (OpenLink Data Spaces) link to start an Personal Editon of OpenLink Data Spaces (or go to: http://your-ec2-instance-cname/dataspace/ods/index.html)
12. Log-in using the username and password credentials for the 'dav' account (or register a new user note: OpenID is an option here also) Create an Data Space Application Instance by clicking on a Data Space App. Tab
13. Import data from your existing Web 2.0 style applications into OpenLink Data Spaces e.g. subscribe to a few RSS/Atom feeds via the "Feeds Manager" application or import some Bookmarks using the "Bookmarks" application
14. Then look at the imported data in Linked Data form via your ODS generated URIs based on the patterns: http://your-ec2-instance-cname/dataspace/person/your-ods-id#this (URI for You the Person), http://your-ec2-instance-cname/dataspace/person/your-ods-id (FOAF File URI), http://your-ec2-instance-cname/dataspace/your-ods-id (SIOC File URI)
    

(OAT) from your Data Space instance

1. http://:8890/oatdemo - Entire OAT Demo Collection
2. http://:8890/rdfbrowser - RDF Browser
3. http://:8890/isparql - SPARQL Query Builder (iSPARQL)
4. http://:8890/qbe - SQL Query Builder (iSQL)
5. http://:8890/formdesigner - Forms Builder (for building Meshups based on RDF, SQL, or Web Servives Data Souces)
6. http://:8890/dbdesigner - SQL DB Schema Designer (note a Visual SQL-RDF Mapper is also on it's way
7. http://:8890/DAV/JS/ - To view the OAT Tree (there are some experimental demos that are missing from the main demo app etc..)

There's more to come!

First off, I am going to focus on the Semantic Data Web aspect of the overall Semantic Web vision (a continuum) as this is what we have now. I am also writing this post as a deliberate contribution to the discourse swirling around the real topic: Semantic Web Value Proposition.

Situation Analysis

We are in the early stages of the long anticipated Knowledge Economy. That being the case, it would be safe to assume that information access, processing, and dissemination are of utmost importance to individuals and organizations alike. You don't produce knowledge in a vacum! Likewise, you can produce Information in a vacum, you need Data.

The Semantic Data Web's value to Individuals

Increasingly, Blogs, Wikis, Shared Bookmarks, Photo Galleries, Discussion Forums, Shared Calendars and the like, have become invaluable tools for individual and organizational participation in Web enabled global discourse (where a lot of knowledge is discovered). These tools, are typically associated with Web 2.0, implying Read-Write access via Web Services, centralized application hosting, and data lock-in (silos).

The reality expressed above is a recipe for "Information Overload" and complete annihilation of ones effective pursuit and exploitation of knowledge due "Time Scarcity" (note: disconnecting is not an option). Information abundance is inversely related to available processing time (for humans in particular). In my case for instance, I was actively subscribed to over 500+ RSS feeds in 2003. As of today, I've simply stopped counting, and that's just my Weblog Data Space. Then add to that, all of the Discussions I track across Blogs, wikis, message boards, mailing lists, traditional usnet discussion forumns, and the like, and I think you get the picture.

Beyond information overload, Web 2.0 data is "Semi-Structured" by way of it's dominant data containers ((X)HTML, RSS, Atom documents and data streams etc.) lacking semantics that formally expose individual data items as distinct entities, endowed with unambiguous naming / identification, descriptive attributes (a type of property/predicate), and relationships (a type of property/predicate).

Devise a standard for Structured Data Semantics that is compatible with the Web Information BUS.

Produce structured data (entities, entity types, entity relationships) from Web 1.0 and Web 2.0 resources that already exists on the Web such that individual entities, their attributes, and relationships are accessible and discernible to software agents (machines).

Once the entities are individually exposed, the next requirement is a mechanism for selective access to these entities i.e. a query language.

Semantic Data Web Technologies that facilitate the solution described above include:

RDF - Data Model for structured data

RDF/XML - A serialization format for RDF based structured data

N3 / Turtle - more human friendly serialization formats for RDF based structured data

GRDDL - enables association between XHTML pages and XSLT stylesheets that facilitates loosely coupled "on the fly" extraction of RDF from non RDF documents

RDFa - enables document publishers or viewers (i.e those repurposing or annotating) to embed structured data into existing XHTML documents

eRDF - another option for embedding structured RDF data within (X)HTML documents

RDF Middleware - typically incorporating GRDDL, RDFa, eRDF, and custom extraction and mapping as part of a structured data production pipeline

Use of URIs or IRIs for uniquely identifying physical (HTML Documents, Image Files, Multimedia Files etc..) and abstract (People, Places, Music, and other abstract things).

SPARQL Query Language - the SQL analog of the Semantic Data Web that enables query constructs that target named entities, entity attributes, and entity relationships

SPARQL Protocol - a REST or SOAP style Web Service for transporting SPARQL Queries to Structured Data Sources.

SPARQL Results Serialization Formats - query results serialization formats that includes XML(sparql+xml) and JSON.

The Semantic Data Web's value to Organizations

Organizations are rife with a plethora of business systems that are built atop a myriad of database engines, sourced from a variety of DBMS vendors. A typical organization would have a different database engine, from a specific DBMS vendor, underlying critical business applications such as: Human Resource Management (HR), Customer Relationship Management (CRM), Accounting, Supply Chain Management etc. In a nutshell, you have DBMS Engines, and DBMS Schema heterogeneity permeating the IT infrastructure of organizations on a global scale, making Data & Information Integration the biggest headache across all IT driven organizations.

Alleviation of the pain (costs) associated with Data & Information Integration.

A dexterous data model (RDF) that enables the construction of conceptual views of disparate data sources across an organization based on existing web architecture components such as HTTP and URIs.

Existing middleware solutions that facilitate the exposure of SQL DBMS data as RDF based Structured Data include:

Virtuoso's Meta Schema Language for RDF Views of SQL Data (also see the Virtuoso SQL-RDF Technical White Paper)

D2RQ

DataGrid

Others

BTW - There is an upcoming W3C Workshop covering the integration of SQL and RDF data.

Conclusion

The Semantic Data Web is here, it's value delivery vehicle is the URI. The URI is a conduit to Interlinked Structured Data (RDF based Linked Data) derived from existing data sources on the World Wide Web alongside data continuously injected into the Web by organizations world wide. Ironically, the Semantic Data Web only platform that crystallizes the: Information at Your Fingertips vision, without development environment, operating system, application, or database lock-in. You simply click on a Linked Data URI and the serendipitous exploration and discovery of data commences.

The unobtrusive emergence of the Semantic Data Web is a reflection of the soundness of the underlying Semantic Web vision.

If you are excited about Mash-ups then your are a Semantic Web enthusiast and benefactor in the making, because you only "Mash" (brute force data extraction and interlinking) because you can't "Mesh" (natural data extraction and interlinking). Likewise, if you are a social-networking, open social-graph, or portable social-network enthusiast, then you are also a Semantic Data Web benefactor and enthusiasts, because your "values" (yes, the values associated with the properties that define you e.g your interests etc) are the fundamental basis for portable, open, social-networking, which is what the Semantic Data Web hands to you on a platter without compromise (i.e. data lock-in or loss of data ownership).

Read/WriteWeb via the OpenLink Data Web Browser (click on the different viewing tabs to see what structured data exploitation in action)

Read/WriteWeb via the Zitgist Data Web Browser

DBpedia (*note: I deliberately use DBpedia URIs in my posts where I would otherwise have used a Wikipedia article URI*)

Zitgist zLinks - Mike Bergman's Blog Post also demonstrating zLinks

Since I am aggressively tracking RDFa developments, I decided to quickly view Ivan's FOAF-in-RDFa file via the OpenLink RDF Browser. The full implications are best understood when you click on each of the Browser's Tabs -- each providing a different perspective on this interesting addition to the Semantic Data Web (note: the Fresnel Tab which demonstrates declarative UI templating using N3).

What's Going on Here?

The OpenLink RDF Browser is a Rich Internet Application built using OAT (OpenLink Ajax Toolkit). In my case, I am deploying the RDF Browser from a Virtuoso instance, which implies that the Browser is able to use the Virtuoso Sponger Middleware (exposed as a REST Service at the Virtuoso instance endpoint: /proxy); which includes an RDFa Cartridge comprised of a metadata extractor and an RDF Schema / OWL Ontology mapper. That's it!

New Semantic Data Web related features and enhancements include:

* A Javascript-based Fresnel processor enabling declarative RDF-based display templates for RDF Data Sources

* An XSLT template for generating HTML pages from the Fresnel processor's XML output

* Interactive Query Builder for SPARQL (iSPARQL). This version of the iSPARQL application includes support for INSERTs and DELETEs

* Enhanced Javascript-based N3/Turtle parser

* New Navigator viewer panel for RDF Browser.

*Project Home Page

*Source Code

*Live Features Demonstrations.

On different, but related, thread, Mike Bergman recently penned a post titled: What is the Structured Web?. Both of these public contributions shed light on the "Information BUS" essence of the World Wide Web by describing the evolving nature of the payload shuttled by the BUS.

What is an Information BUS?

Middleware infrastructure for shuttling "Information" between endpoints using a messaging protocol.

The Web is the dominant Information BUS within the Network Computer we know as the "Internet". It uses HTTP to shuttle information payloads between "Data Sources" and "Information Consumers" - what happens when we interact with Web via User Agents / Clients (e.g Browsers).

What are Web Information Payloads?

HTTP transported streams of contextualized data. Hence the terms: "Information Resource" and "Non Information" when reading material related to http-range-14 and Web Architecture. For example, an (X)HTML document is a specific data context (representation) that enables us to perceive, or comprehend, a data stream originating from a Web Server as a Web Page. On the other hand, if the payload lacks contextualized data, a fundamental Web requirement, then the resource is referred to as a "Non Information" resource. Of course, there is really no such thing as a "Non Information" resource, but with regards to Web Architecture, it's the short way of saying: "the Web Transmits Information only". That said, I prefer to refer to these "Non Information" resources as "Data Sources", are term well understood in the world of Data Access Middleware (ODBC, JDBC, OLEDB, ADO.NET etc.) and Database Management Systems (Relational, Objec-Relational, Object etc).

Examples of Information Resource and Data Source URIs:

http://demo.openlinksw.com/Northwind/Customer/ALFKI (Information Resource)

http://demo.openlinksw.com/Northwind/Customer/ALFKI#this (Data Source)

Explanation: The Information Resource is a conduit to the Entity identified by Data Source (an entity in my RDF Data Space that is the Subject or Object of one of more Triple based Statements. The triples in question can that can be represented as an RDF resource when transmitted over the Web via an Information Resource that takes the form of a SPARQL REST Service URL or a Physical RDF based Information Resource URL).

What about Structured Data?

Prior to the emergence of the Semantic Data Web, the payloads shuttled across the Web Information BUS comprised primarily of the following:

1. HTML - Web Resource with presentation focused structure (Web 1.0 dominant payload form)
2. XML - Web Resource with structure that separates presentation and data (Web 2.0's dominant payload form).

The Semantic Data Web simply adds RDF to the payload formats that shuttle the Web Information BUS. RDF addresses formal data structure which XML doesn't cover since it is semi-structured (distinct data entities aren't formally discernible). In a nutshell, an RDF payload is basically a conceptual model database packaged as an Information Resource. It's comprised of granular data items called "Entities", that expose fine grained properties values, individual and/or group characteristics (attributes), and relationships (associations) with other Entities.

Where is this all headed?

The Web is in the final stages of the 3rd phase of it's evolution. A phase characterized by the shuttling of structured data payloads (RDF) alongside less data oriented payloads (HTML, XHTML, XML etc.). As you can see, Linked Data and Structured Data are both terms used to describe the addition of more data centric payloads to the Web. Thus, you could view the process of creating a Structured Web of Linked Data as follows:

1. Identify or Create Structured Data Sources
2. Name these Data Sources using Data Source URIs
3. Expose Structured Data Sources to the Web as Linked Data using Information Resource (conduit) URIs

Conclusions

The Semantic Data Web is an evolution of the current Web (an Information Space) that adds structured data payloads (RDF) to current, less data oriented, structured payloads (HTML, XHTML, XML, and others).

The Semantic Data Web is increasingly seen as an inevitability because it's rapidly reaching the point of critical mass (i.e. network effect kick-in). As a result, Data Web emphasis is moving away from: "What is the Semantic Data Web?" To: "How will Semantic Data Web make our globally interconnected village an even better place?", relative to the contributions accrued from the Web thus far. Remember, the initial "Document Web" (Web 1.0) bootstrapped because of the benefits it delivered to blurb-style content publishing (remember the term electronic brochure-ware?). Likewise, in the case of the "Services Web" (Web 2.0), the bootstrap occurred because it delivered platform independence to Web Application Developers - enabling them to expose application logic behind Web Services. It is my expectation that the Data Integration prowess of the Data Web will create a value exchange realm for data architects and other practitioners from the database and data access realms.

Related Items

1. Mike Bergman's post about Semi-Structured Data
2. My Posts covering Structured and Un-Structured Containers

Now that broader understanding of the Semantic Data Web is emerging, I would like to revisit the issue of "Data Spaces".

A Data Space is a place where Data Resides. It isn't inherently bound to a specific Data Model (Concept Oriented, Relational, Hierarchical etc..). Neither is it implicitly an access point to Data, Information, or Knowledge (the perception is purely determined through the experiences of the user agents interacting with the Data Space.

A Web Data Space is a Web accessible Data Space.

Real world example:

Today we increasing perform one of more of the following tasks as part of our professional and personal interactions on the Web:

1. Blog via many service providers or personally managed weblog platforms
2. Create Event Calendars via Upcoming.com and Eventful
3. Maintain and participate in Social Networks (e.g. Facebook, Orkut, MySpace)
4. Create and Participate in Discussions (note: when you comment on blogs or wikis for instance, you are participating in, or creating, a conversation)
5. Track news by subscribing to RSS 1.0, RSS 2.0, or Atom Feeds
6. Share Bookmarks & Tags via Del.icio.us and other Services
7. Share Photos via Flickr
8. Buy, Review, or Search for books via Amazon
9. Participates in auctions via eBay
10. Search for data via Google (of course!)

John Breslin has nice a animation depicting the creation of Web Data Spaces that drives home the point.

Unfortunately, what isn't as obvious to many netizens, is the fact that each of the activities above results in the creation of data that is put into some context by you the user. Even worse, you eventually realize that the service providers aren't particularly willing, or capable of, giving you unfettered access to your own data. Of course, this isn't always by design as the infrastructure behind the service can make this a nightmare from security and/or load balancing perspectives. Irrespective of cause, we end up creating our own "Data Spaces" all over the Web without a coherent mechanism for accessing and meshing these "Data Spaces".

Data Spaces on the Web that provide granular access to RDF Data.

Short History

In anticipation of this the "Web Data Silo" challenge (an issue that we tackled within internal enterprise networks for years) we commenced the development (circa. 2001) of a distributed collaborative application suite called OpenLink Data Spaces (ODS). The project was never released to the public since the problems associated with the deliberate or inadvertent creation of Web Data silos hadn't really materialized (silos only emerged in concreted form after the emergence of the Blogosphere and Web 2.0). In addition, there wasn't a clear standard Query Language for the RDF based Web Data Model (i.e. the SPARQL Query Language didn't exist).

Today, ODS is delivered as a packaged solution (in Open Source and Commercial flavors) that alleviates the pain associated with Data Space Silos that exist on the Web and/or behind corporate firewalls. In either scenario, ODS simply allows you to create Open and Secure Data Spaces (via it's suite of applications) that expose data via SQL, RDF, XML oriented data access and data management technologies. Of course it also enables you to integrates transparently with existing 3rd party data space generators (Blogs, Wikis, Shared Bookmrks, Discussion etc. services) by supporting industry standards that cover:

1. Content Publishing - Atom, Moveable Type, MetaWeblog, Blogger protocols
2. Content Syndication Formats - RSS 1.0, RSS 2.0, Atom, OPML etc.
3. Data Management - SQL, RDF, XML, Free Text
4. Data Access - SQL, SPARQL, GData, Web Services (SOAP or REST styles), WebDAV/HTTP
5. Semantic Data Web Middleware - GRDDL, XSLT, SPARQL, XPath/XQuery, HTTP (Content Negotiation) for producing RDF from non RDF Data ((X)HTML, Microformats, XML, Web Services Response Data etc).

Thus, by installing ODS on your Desktop, Workgroup, Enterprise, or public Web Server, you end up with a very powerful solution for creating Open Data access oriented presence on the "Semantic Data Web" without incurring any of the typically assumed "RDF Tax".

Naturally, ODS is built atop Virtuoso and of course it exploits Virtuoso's feature-set to the max. It's also beginning to exploit functionality offered by the OpenLink Ajax Toolkit (OAT).

Shelley Powers via her post titled: More Ajax Security and many others.

In anticipation of the obvious concerns of many Javascript based developers, Ondrej Zara (lead developer of the OpenLink Ajax Toolkit) has written a post titled: OAT and JS Hijacking, that explains the security aspects our Javascript Toolkit in relation to this alert

From my perspective on things I prefer to align my articulation of the changes that are occurring across our industry (courtesy of the Internet Inflection) to the MVC pattern.

Re. the Web Versions (or Dimensions of Interaction):

Web 1.0 - (V)iewer (Interactive Web experienced via Browser)

Web 2.0 - (C)ontroller Web (via Web Services API)

Web 3.0 - (M)odel (via the RDF Data Model as the basis for an Open and Standards based Concrete Conceptual Data Model)

The same applies to evolution of Openness:

Early work by Sun and other early UNIX Vendors - (V)iewer (Interaction with the same OS across different hardware platforms)

Open Source Movement - (C)ontroller (Open Access to Application Source Code )

Open Data - (M)odel (*where we are now* Freeing the Date from the Applications and Services while moving the application development focus to a Concrete Conceptual Data Model focus. The Data Web is a classic example.)

In the (C)ontroller realm where the focal point is Application Logic, data access issues aren't obvious (*I recall my battles with Richard Stallman re. the appropriate Open Source License variant for iODBC during the embryonic years of database and data access technology on Linux*). Data is an enigma in this realm, unfortunately. This implies that "Data Lock-in" occurs deliberately, but in most cases, inadvertently when we make Application Logic the focal point of everything. Another example is Web 2.0 in which the norm (unfortunately) is to suck in your data, and then refuse to give you complete ownership over how it is used (including the fact that you may want to share it elsewhere).

Some Definitions (as per usual):

RDF Middleware (as defined in this context) is about producing RDF from non RDF Data Sources. This implies that you can use non RDF Data Sources (e.g. (X)HTML Web Pages, (X)HTML Web Pages hosting Microformats, and even Web Services such as those from Google, Del.icio.us, Flickr etc..) as Semantic Web Data Source URIs (pointers to RDF Data).

In this post I would like to provide a similar perspective on this ability to treat non RDF as RDF from RDF Browser perspective.

First off, what's an RDF Browser?

An RDF Browser is a piece of technology that enables you to Browse RDF Data Sources by way of Data Link Traversal. The key difference between this approach and traditional browsing is that Data Links are typed (they possess inherent meaning and context) whereas traditional links are untyped (although universally we have been trained to type them as links to Blurb in the form of (X)HTML pages or what is popularly called "Web Content".).

There are a number of RDF Browsers that I am aware off (note: pop me a message directly of by way of a comment to this post if you have a browser that I am unaware of), and they include (in order of creation and availability):

1. Tabulator
2. DISCO - Hyperdata Browser
3. OpenLink Ajax Toolkit's RDF Browser (a component of the OAT Javascript Toolkit)

Each of the browsers above can consume the services of Triplr or the Virtuoso Sponger en route to unveiling a RDF Data that is traversable via URI dereferencing (HTTP GETing the data exposed by the Data Pointer). Thus you can cut&paste the following into each of the aforementioned RDF Browsers:

1. Triplr's RDF Data (Triples) extractions from Dan Connolly's Home Page
2. The Virtuoso Sponger's RDF Data (Triples) extractions from Dan Connolly's Home Page

Since we are all time challenged (naturally!) you can also just click on these permalinks for the OAT RDF Browser demos:

1. Permalink for Triplr's RDF Data (Triples) extractions from Dan Connolly's Home Page
2. Permalink for the Virtuoso Sponger's RDF Data (Triples) extractions from Dan Connolly's Home Page

Quick Definitions:

Mashups - Brute force joining of disparate Web Data

Meshups - Natural joining of disparate Web Data

Reasons for the distinction:

Mashups are Data Model oblivious.

Meshups are Data Model driven.

Examples:

Mashups are based on RSS 2.0 most of the time (RSS 2.0 is at best a Tree Structure that contains untyped or meaning challenged links.

Meshups are RDF based and the data is self describing since the links are typed (posses inherent meaning thereby providing context).

So what? You may be thinking.

For starters, I can quite easily Mesh data from Googlebase (which emits RSS 2.0 or Atom) and other data sources with the Mapping Services from Yahoo!

I can achieve this in minutes without writing a single line of code. I can do it because of the Data Model prowess of RDF (self-describing instance-data), the data interchange and transformation power of XML and XSLT respectively, the inherent power of XML based Web Services (REST or SOAP), and of course, having a Hybrid Server product like Virtuoso at my disposal that delivers a cross platform solution for exploiting all of these standards coherently.

I can share the self-describing describing data source that serves my Meshup. Try reusing the data presented by a Mashup via the same URL that you used to locate Mashup to get my drift.

Demo Links:

1. Googlebase Query URL as an RDF Data Source
2. Perform a simple Data Mesh by adding (via link copy and paste) this Upcoming.org Query Services URL for Ajax Events to the RDF Browsers list of Data Sources (paste into the Data Source URI input field).

What does this all mean?

"Context" is the catalyst of the burgeoning Data Web (Semantic Web Layer - 1). It's the emerging appreciation of "Context" that is driving the growing desire to increment Web versions from 2.0 to 3.0. It also the the very same "Context" that has been a preoccupation of Semantic Web vision since its inception.

The journey towards a more Semantic Web is all inclusive (all "ANDs" and no "ORs" re. participation).

The Semantic Web is self-annotating. Web 2.0 has provided a huge contribution to the self annotation effort: on the Web we now have Data Spaces for Bookmarks (e.g del.icio.us), Image Galleries ( e.g Flickr), Discussion Forums (remember those comments associated with blog posts? ditto the pingbacks and trackbacks?), People Profiles (FOAF, XFN, del.icio.us, and those crumbling walled-gardens around many Social Networks), and more..

A Web without granular access to Data is simply not a Web worth having (think about the menace of click-fraud and spam).

We are hitting a little problem re. Web 3.0 and Web 2.0, naturally :-) Web 2.0 is one of several (present and future) Dimensions of Web Interaction that turns Web Sites into Web Services Endpoints; a point I've made repeatedly [1] [2] [3] [4] across the blogosphere, in addition to my early futile attempts to make the Wikipedia's Web 2.0 article meaningful (circa 2005), as per the Wikipedia Web 2.0 Talk Page excerpt below:

Web 2.0 is a web of executable endpoints and well formed content. The executable endpoints and well formed content are accessible via URIs. Put differently, Web 2.0 is a web defined by URIs for invoking Web Services and/or consuming or syndicating well formed content.

Hopefully, someone with more time on their hands will expand on this ( I am kinda busy)

.

BTW - Web 2.0 being a platform doesn't distinguish it in anyway from Web 1.0. They are both platforms, the difference comes down to platform focus and mode of experience.

Web 3.0 is about Data Spaces: Points of Semantic Web Presence that provide granular access to Data, Information, and Knowledge via Conceptual Data Model oriented Query Languages and/or APIs.

The common denominator across all the current and future Web Interaction Dimensions is HTTP. While their differences are as follows:

Web 1.0 - Browser (HTTP + (X)HTML)

Web 2.0 - Presence (Web Service Endpoints for REST or SOAP over HTTP)

Web 3.0 - Presence (Query Languages, Data Models, and HTTP based Query Oriented Web Service Endpoints)

Examples of Web 3.0 Infrastructure:

1. Query Languages: SPARQL, Googlebase Query Language, Facebook Query Language (FQL), and many others to come
2. Query Language aligned Web Services (Query Services): SPARQL Protocol, GData, or REST style Web services such as Facebook's service for FQL.
3. Data Models: Concrete Conceptual Data Model (which RDF happens to deliver for Web Data)

Web 3.0 is not purely about Web Sites becoming Web Services endpoints. It is about the "M" (Data Model) taking it's place in the MVC pattern as applied to the Web Platform.

I will repeat myself yet again:

The Devil is in the Details of the Data Model. Data Models make or break everything. You ignore data at your own peril. No amount of money in the bank will protect you from Data Ignorance! A bad Data Model will bring down any venture or enterprise, the only variable is time (where time is directly related to your increasing need to obtain, analyze, and then act on data, over repetitive operational cycles, that have ever decreasing intervals).

This applies to the Real-time enterprise of Information and/or knowledge workers and Real-time Web Users alike.

BTW - Data Makes Shifts Happen (spotter: Sam Sethi).

Web 2.0 commentators such as Mike Arrington, and as mentioned above,Tim O'Reilly, both blogged about the imminent release of Freebase earlier today. Although I haven't looked at this database yet, it is crystal clear to me that it is one of many Web Databases to come. Others that I am personally familiar with, and involved in, include: DBpedia (Wikipedia as a true Database) and Zitgist (soon to be unveiled).

All of these databases mark the crystallization of the "Data Web" and the imminence of what is increasingly referred to as Web 3.0.

I certainly hope that all web 3.0 Database Providers keep the data Open, adhere to Web Best Practice recipes for sharing and publishing data, and generally make the process of data, information, and knowledge discovery via the Web much easier.

Linking personal posted content across communities: "

With the help of Kingsley, Uldis and I have been looking at how SIOC can be used to link the content that a single person posts to a number of community sites. The picture below shows an example of stuff that I’ve created on Flickr, YouTube, etc. through my various user identities on those sites (these match some SIOC types that we want to add to a separate module). We can also say that each Web 2.0 content item is a user-contributed post, with some attached or embedded content (e.g. a file or maybe just some metadata). This is part of a new discussion on the sioc-dev mailing list, and we’d value your contributions.

Edit: The inner layer is a person (semantically described in FOAF), the next layer is their user accounts (described in FOAF, SIOC) and the outer layer is the posted content - text, files, associated metadata - on community sites (again described using SIOC).

(Via John Breslin - Cloudlands.)

The point that John is making about the Data Web and Interlinked Data Spaces exposed via URIs (e.g Personal URIs), crystallizes a number of very important issues about the Data Web that may remain unclear. I am hoping that by digesting the post excerpt above, in conjunction with the items below, aids the pursuit of clarity and comprehension about the all important Data Web (Semantic Web - Layer 1):

1. Your OpenID can be Your Personal URI (as noted by Henry Story's post about: The Many Uses of OpenID). That that's what I have courtesy of OpenLink Data Spaces (ODS)
2. The above only works unobtrusively (i.e. OpenID and Personal sharing a URI) if Content Negotiation is exploited on the Client and Server sides.
3. TimBL's call out to Share Your Data and Link to Other Data via URIs via post titled: Give Yourself a URI.
4. W3C's Best Practice Recipes for Publishing RDF Vocabularies
5. W3C's Architecture of the World Wide Web - Vol 1 which covers URI Dereferencing (HTTP GET-ing the data that a URI points to)
6. Richard Cyganiak's post titled: Debugging Semantic Web Sites with Curl.

Examples of some of these principles in practice:

1. Chris Bizer, Tobias Gaub, and Richard's Javascript based Semantic Web Client Library
2. DISCO RDF Browser
3. OpenLink Ajax Toolkit's (OAT) RDF Browser
4. OpenLink Interactive SPARQL Query by Example (iSPARQL QBE)
5. Dynamic Data Web Pages from my prior posts [1][2][3]
6. dbpedia (Wikipedia as a Data Web oriented Data Source)
7. And of course this blog post's permalink is a bona fide dereferencable URI.

And of course there is more to come such as Grandma's Semantic Web Browser which is coming from Zitgist LLC (pronounced: Zeitgeist) a joint venture of OpenLink Software and Frederick Giasson.

Here is what I was I was able to knock together using my SPARQL QBE (without writing the SPARQL by hand):

1. Forest Whitaker Data
2. Helen Mirren Data
3. Jennifer Hudson Data.

Note: Just select the "Explore" option when the link-lookup window appears in response to you clicking on any of the links. That said, if you are using the Firefox Linkification extension the page will not work properly (as per this discussion about disabling Linkification) :-(

BTW - I have a comments page, so don't be shy about showing me how you could produce this kind of data driven web page much quicker than I have :-)

Warning: IE6 and Safari (use Webkit instead) cannot process these pages due to the use of Ajax.

In this third take on my introduction to the Data Web I would like to share a link with you (a Dynamic Start Page in Web 2.0 parlance) with a Data Web twist: You do not have to preset the Start Page Data Sources (this is a small-big thing, if you get my drift, hopefully!).

Here are some Data Web based Dynamic Start Pages that I have built for some key play ers from the Semantic Web realm (in random order):

1. Dan Brickley
2. Tim Berners-Lee
3. Dan Connolly
4. Danny Ayers
5. Planet RDF

"These are RDF prepped Data Sources....", you might be thinking, right? Well here is the reminder: The Data Web is a Global Data Generation and Integration Effort. Participation may be active (Semantic Web & Microformats Community), or passive (web sites, weblogs, wikis, shared bookmarks, feed subscription, discussion forums, mailing lists etc..). Irrespective of participation mode, RDF instance can be generated from close to anything (I say this because I plan to add binary files holding metadata to this mix shortly). Here are examples of Dynamic Start Pages for non RDF Data Sources:

1. del.icio.us Web 2.0 Events Bookmarks
2. Vecosys
3. Techcrunch
4. Jon Udell's Blog
5. Dave Winer's Scripting News
6. Robert Scoble's Blog

what about Microformats you may be wondering? Here goes:

1. Microformats Wiki (click on the Brian Suda link for instance)
2. Microformats Planet
3. Del.icio.us Microformats Bookmarks
4. Ben Adida's home page (RDFa)

Let's carry on.

How about some traditional Web Sites? Here goes:

1. OpenLink Software's Home Page
2. Oracle's Home Page
3. Apple's Home Page
4. Microsoft's Home Page
5. IBM's Home Page

And before I forget, here is My Data Web Start Page .

Due to the use of Ajax in the Data Web Start Pages, IE6 and Safari will not work. For Mac OS X users, Webkit works fine. Ditto re. IE7 on Windows.

Play Date: What is that thing on the Wall?
My Son: Security Alarm
Play Date: How does it work
My Son: If you click on that top button and then open the door, I will have to enter a code when we come back in or the alarm will go off
Play Date: What is the code?
My Son: I can't tell you that!
Play Date: Why not?
My Son: You might come and steal something from our house!
Play Date: No I won't!
My Son: Well, you might tell someone that might come and steal something from our house! or that person could tell someone who could tell someone that would steal from our house

LOL!! of course! At the same time wondering, how come a majority of adults don't quite see the need for granular access to Web Data in a manner that enables computers and humans to collectively arrive at similar decisions?

Putting Data in context en route to producing actionable knowledge is a transient endeavor that engages a myriad of human senses. We demonstrate comprehension of this fact in our daily existence as social creatures (at a very early age as depicted above). That said, we seem to forget this fact when engaging the Web: If we can't see it then it can't be valuable.

BTW - I just received a ping about the "Sensory Web" (which is just another way of describing a Data Driven Web experience from my vantage point.)

In the popular M-V-C pattern you don't see the "M", but the "M" will kill you if you get it wrong (it is the FORCE)! Coming to think about it, the pattern could have been coined: V-C-M or C-M-V, but isn't for obvious reasons :-)

RDF is the vehicle that enables us tap into the Data aspect of the Web. We started off with pages of blurb linked via hypertext (Web 1.0) and then looked to "Keywords" for some kind of data access; we then isolated some "Verbs" and discovered another dimension of Web Interaction (Web 2.0) but looked to these "Verbs" for data access which left us with Mashups; and now we are starting to extract "Nouns" and "Adjectives" from sentences (Subject, Predicate, Object - Triples) associated with resources on the Web (Data Web / Web 3.0 / Semantic Web Layer 1) which provides a natural data access substrate for Meshups (natural joining of disparate data from a plethora of data sources) while providing the foundation layer for the Semantic Web.

For those who need use-cases that demonstrate tangible value re. the Semantic Web, here are some projects to note courtesy of the Semantic Web Education and Outreach (SWEO) interest group:

1. FOAF based White-lists - Attacking SPAM
2. Open Data Access and Linking for the Data Web - Data Integration and Generation effort that creates a cluster of RDF instance data from a myriad of data sources relating to every day things such as: People, Places, Events, Projects, Discussions, Music, Books, and other things
3. Content Labeling - Protecting our kids on the Web amongst other matters relating to knowledge about data sources
4. Others..

1. Data Web and Global Data Integration & Generation Effort
2. Previous Data Web posts.

1. go to http://demo.openlinksw.com/isparql
2. Enter any of the following values into the "Default Data URI"; field:
- http://www.mkbergman.com/?p=336
- http://radar.oreilly.com/archives/2007/02/pipes_and_filte.html
- http://jeremy.zawodny.com/blog/archives/008513.html
- Other URIs

What I am demonstrating is how existing Web Content hooks transperently into the "Data Web". Zero RDF Tax :-) Everything is good!

Note: Please look to the bottom of the screen for the "Run Query" Button. Remember, it not quite Grandma's UI but should do for Infonauts etc.. A screencast will follow.

OAT: OpenAjax Alliance Compliant Toolkit: "

Ondrej Zara and his team at Openlink Software have created a Openlink Software JS Toolkit, known as OAT. It is a full-blown JS framework, suitable for developing
rich applications with special focus to data access.

OAT works standalone, offers vast number of widgets and has some rarely seen features, such as on-demand library loading (which reduces the total amount of downloaded JS code).

OAT is one of the first JS toolkits which show full OpenAjax Alliance conformance: see the appropriate wiki page and conformance test page.

There is a lot to see with this toolkit:

You can see some of the widgets in a Kitchen sink application

Sample data access applications:

• SQL Query By Example
• Forms designer
• DB Designer

OAT is Open Source and GPL’ed over at sourceforge and the team has recently managed to incorporate our OAT data access layer as a
module to dojo datastore.

(Via Ajaxian Blog.)

This is a corrected version of the initial post. Unfortunately, the initial post was inadvertently littered with invalid links :-( Also, since the original post we have released OAT 1.2 that includes integration of our iSPARQL QBE into the OAT Form Designer application.

Re. Data Access, It is important to note that OAT's Ajax Database Connectivity layers supports data binding to the following data source types:

1. RDF - via SPARQL (Query Language, Protocol, and Resultset Serialization formats: RDF/XML, RDF/N3, RDF/Turtle, XML, and JSON)
2. SQL - via XMLA (somewhat forgotten SOAP protocol for SQL Data Access that can sit atop ODBC, ADO.NET, OLE-DB, and even JDBC)
3. XML - via SOAP or REST style Web Services

1. Tabular Grids
2. Pivot Tables
3. TimeLines
4. Extended Anchor Tags
5. Map Service Controls (Google, Yahoo!, OpenLayers, Microsoft Visual Earth)
6. SVG based RDF Graph Control (Opera 9.x provides best viewing experience at the current time)

OAT also includes a number of prototype applications that are completely developed using OAT Controls and Libraries:

1. Visual SPARQL Query Builder
2. Visual SQL Query Builder
3. Web Forms Designer (includes Drag-Drop usage of Data Aware Controls etc.)
4. Visual DB Designer

Note: Pick "Local DSN" from page initialization dialog's drop-down list control when prompted

Dare:

I have been through the Wikipedia fires a few times. If you recall that I actually triggered the early Web 2.0 Wikipedia article. along the following lines:

1. Asked one of my staff to start a post with the sole intention of defining Web 2.0 properly
2. I then attempted to edit the initial post
3. I left a typo re. REST
4. Got set on Fire etc... (see very beginning of Wikipedia Web 2.0 history page)

As annoying as the experience above was, I didn't find this inconsistent with the spirit of Wikipedia (i.e. open contribution and discourse). I felt, at the time, that a lot of historical data was being left in place for future reference etc.. In addition, the ultimate aim of creating an evolving Web 2.0 document did commence albeit some distance from "modern man" re. accuracy and meaningfulness as of my last read (today).

Even closer to home, I repeated the process above re. Virtuoso Universal Server. This basically ended up being a live case study on how you handle the Wikipedia NPOV conundurum. Just look at the Virtuoso Universal Server Talk Pages to see how the process evolved (the key was Virtuoso's lineage and it's proximity to the very DBMS platform upon which Wikipedia runs i.e MySQL).

Bearing in mind the size and magnitude of Microsoft, there should be no reason why Microsoft's "Microsoft Digital Caucus" ( legions of Staff, MSDN members, Integrators, and other partners) can't simply go into Wikipedia and participate in the edit and discourse process.

Truth cannot be surpressed! At best, it can only be temporarily delayed :-) Even more so on the Web!

SPARQL (query language for the Semantic Web) basically enables me to query a collection of typed links (predicates/properties/attributes) in my Data Space (ODS based of course) without breaking my existing local bookmarks database or the one I maintain at del.icio.us.

I am also demonstrating how Web 2.0 concepts such as Tagging mesh nicely with the more formal concepts of Topics in the Semantic Web realm. The key to all of this is the ability to generate RDF Data Model Instance Data based on Shared Ontologies such as SIOC (from DERI's SIOC Project) and SKOS (again showing that Ontologies and Folksonomies are complimentary).

This demo also shows that Ajax also works well in the Semantic Web realm (or web dimension of interaction 3.0) especially when you have a toolkit with Data Aware controls (for SQL, RDF, and XML) such as OAT (OpenLink Ajax Toolkit). For instance, we've successfully used this to build a Visual Query Building Tool for SPARQL (alpha) that really takes a lot of the pain out of constructing SPARQL Queries (there is much more to come on this front re. handling of DISTINCT, FILTER, ORDER BY etc..).

For now, take a look at the SPARQL Query dump generated by this SIOC & SKOS SPARQL QBE Canvas Screenshot.

You can cut and paste the queries that follow into the Query Builder or use the screenshot to build your variation of this query sample. Alternatively, you can simply click on *This* SPARQL Protocol URL to see the query results in a basic HTML Table. And one last thing, you can grab the SPARQL Query File saved into my ODS-Briefcase (the WebDAV repository aspect of my Data Space).

Note the following SPARQL Protocol Endpoints:

1. MyOpenLink Data Space
2. Experimental Data Space SPARQL Query Builder (you need to register at http://myopenlink.net:8890/ods to use this version)
3. Live Demo Sever
4. Demo Server SPARQL Query Builder (use: demo for both username and pwd when prompted)

My beautified Version of the SPARQL Generated by QBE (you can cut and paste into "Advanced Query" section of QBE) is presented below:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX dct: <http://purl.org/dc/elements/1.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>


SELECT distinct 
       ?forum_name, 
       ?owner, 
       ?post, 
       ?title, 
       ?link, 
       ?url, 
       ?tag
FROM <http://myopenlink.net/dataspace>
WHERE {
       ?forum a sioc:Forum;
                   sioc:type "bookmark";
                   sioc:id ?forum_name;
                   sioc:has_member ?owner.
       ?owner sioc:id "kidehen".
       ?forum sioc:container_of ?post .
       ?post  dct:title ?title .
       optional { ?post sioc:link ?link  }
       optional { ?post sioc:links_to ?url }
       optional { ?post sioc:topic ?topic.
                        ?topic a skos:Concept;
                                  skos:prefLabel ?tag}.
     } 

Unmodified dump from the QBE (this will be beautified automatically in due course by the QBE):

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX dct: <http://purl.org/dc/elements/1.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>


SELECT ?var8 ?var9 ?var13 ?var14 ?var24 ?var27 ?var29 ?var54 ?var56
WHERE
{
graph ?graph {
 ?var8 rdf:type sioc:Forum .
 ?var8 sioc:container_of ?var9 .
 ?var8 sioc:type "bookmark" .
 ?var8 sioc:id ?var54 .
 ?var8 sioc:has_member ?var56 .
 ?var9 rdf:type sioc:Post .
 OPTIONAL {?var9 dc:title ?var13} .
 OPTIONAL {?var9 sioc:links_to ?var14} .
 OPTIONAL {?var9 sioc:link ?var29} .
 ?var9 sioc:has_creator ?var37 .
 OPTIONAL {?var9 sioc:topic ?var24} .
 ?var24 rdf:type skos:Concept .
 OPTIONAL {?var24 skos:prefLabel ?var27} .
 ?var56 rdf:type sioc:User .
 ?var56 sioc:id "kidehen" .
 }
} 

Current missing items re. Visual QBE for SPARQL are:

1. Ability to Save properly to WebDAV so that I can then expose various saved SPARQL Queries (.rq file) from my Data Space via URIs
2. Handling of DISTINCT, FILTERS (note: OPTIONAL is handled via dotted predicate-links)
3. General tidying up re. click event handling etc.

1. Export some bookmarks from your local browser to the usual HTML bookmarks dump file
2. Create an ODS-Bookmarks Instance using your new ODS account
3. Use the ODS-Bookmark Instance to import your local bookmarks from the HTML dump file
4. Repeat the same import sequence using the ODS-Bookmark Instance, but this time pick the del.icio.us option
5. Build your query (change 'kidehen' to your ODS-user-name)
6. That's it you now have Semantic Web presence in the form of a Data Space for your local and del.icio.us hosted bookmarks with tags integrated

Quick Query Builder Tip: You will need to import the following (using the Import Button in the Ontologies & Schemas side-bar);

1. http://www.w3.org/1999/02/22-rdf-syntax-ns# (RDF)
2. http://rdfs.org/sioc/ns# (SIOC)
3. http://purl.org/dc/elements/1.1/ (Dublin Core)
4. http://www.w3.org/2004/02/skos/core# (SKOS)

Browser Support: The SPARQL QBE is SVG based and currently works fine with the following browsers; Firefox 1.5/2.0, Camino (Cocoa variant of Firefox for Mac OS X), Webkit (Safari pre-release / advanced sibling), Opera 9.x. We are evaluating the use of the Adobe SVG plugin re. IE 6/7 support.

Of course this should be a screencast, but I am the middle of a plethora of things right now :-)

I came across a pretty deep comments trail about the aforementioned items on Fred Wilson's blog (aptly titled: A VC) under the subject heading: Web 3.0 Is The Semantic Web.

Contributions to the general Semantic Web discourse by way of responses to valuable questions and commentary contributed by a Semantic Web skeptic (Ed Addison who may be this Ed Addison according to Google):

Ed, Responses to your points re. Semantic Web Matrialization:

<< 1) ontologies can be created and maintained by text extractors and crawlers" >>

Ontologies will be developed by Humans. This process has already commenced and far more landscape has been covered that you may be aware of. For instance, there is an Ontology for Online Communities with Semantics factored in. More importantly, most Blogs, Wikis, and other "points of presence" on the Web are already capable of generating Instance Data for this Ontology by way of the underlying platforms that drive these things. The Ontology is called: SIOC (Semantically-Interlinked Online Communities).

<< 2) the entire web can be marked up, semantically indexed, and maintained by spiders without human assistance >>

Most of it can, and already is :-) Human assistance should, and would, be on an "exception basis" a preferred use of human time (IMHO). We do not need to annotate the Web manually when this labor intensive process can be automated (see my earlier comments).

<< 3) inference over the semantic web does not require an extremely deep heuristic search down multiple, redundant, cyclical pathways with many islands that are disconnected >>

When you have a foundation layer of RDF Data (generated in the manner I've discussed above), you then have a substrate that's far more palatable to Intelligent Reasoning. Note, the Semantic Web is made of many layers. The critical layer at this juncture is the Data-Web (Web of RDF Data). Note, when I refer to RDF I am not referring to RDF/XML the serialization format, I am referring to the Data Model (a Graph).

<< 4) the web becomes smart enough to eliminate websites or data elements that are incorrect, misleading, false, or just plain lousy >>

The Semantic Web vision is not about eliminating Web Sites (The Hypertext-Document-Web). It is simply about adding another dimension of interaction to the Web. This is just like the Services-Web dimension as delivered by Web 2.0.

We are simply evolving within an innovation continuum. There is no mutual exclusivity about any of the Web Dimensions since they collectively provide us with a more powerful infrastructure for building and exploiting "collective wisdom".

As for the Data-Web experiment part of this post, I would expect to see this post exposed as another contribution to the Data-Web via the PingTheSemanticWeb notification service :-) Implying, that all the relevant parts of this conversation are in a format (Instance Data for the SIOC Ontology) that is available for further use in a myriad of forms.

Web Me2.0 -- Exploding the Myth of Web 2.0:

"Many people have told me this week that they think 'Web 2.0' has not been very impressive so far and that they really hope for a next-generation of the Web with some more significant innovation under the hood -- regardless of what it's called. A lot of people found the Web 2.0 conference in San Francisco to be underwhelming -- there was a lot of self-congratulation by the top few brands and the companies they have recently bought, but not much else happening. Where was all the innovation? Where was the focus on what's next? It seemed to be a conference mainly about what happened in the last year, not about what will happen in the coming year. But what happened last year is already so 'last year.' And frankly Web 2.0 still leaves a lot to be desired. The reason Tim Berners-Lee proposed the Semantic Web in the first place is that it will finally deliver on the real potential and vision of the Web. Not that today's Web 2.0 sucks completely -- it only sort of sucks. It's definitely useful and there are some nice bells and whistles we didn't have before. But it could still suck so much less!"

Web 2.0 is a (not was) a piece of the overall Web puzzle. The Data Web (so called Web 3.0) is another critical piece of this puzzle, especially as it provides the foundation layer (Layer 1) of the Semantic Web.

Web 2.0 was never about "Open Data Access", "Flexible Data Models", or "Open World" meshing of disparate data sources built atop disparate data schemas (see: Web 2.0's Open Data Access Conundrum). It was simply about "Execution and APIs". I already written about "Web Interaction Dimensions", but you call also look at the relationship of the currently perceived dimensions through the M-V-C programming pattern:

1. Viewer (V) - Web 1.0 (Interaction, Dimension 1 - Interactive-Web)
2. Controller (C) - Web 2.0 (Services, Dimension 2 - Services-Web which is about Execution & Application Logic; SOA outside/in-front-of the Firewall for Enterprise 2.0 crowd)
3. Model (M) - Web 3.0 (Data, Dimension 3 - Data-Web which is about data model dexterity and open data access)

Another point to note, Social Networking is hot, but nearly every social network that I know (and I know and use most of them) suffers from an impedance mismatch between the service(s) they provide (social networks) and their underlying data models (in many cases Relational as opposed to Graph). Networks are about Relationships (N-ary) and your cannot effectively exploit the deep potential of: "Network Effects" (Wisdom of Crowds, Viral Marketing etc..) without a complimentary data model, you simply can't.

Finally, the Data Web is already here, I promised a long time ago (Internet Time) that the manifestation of the Semantic Web would occur unobtrusively, meaning, we will wake up one day and realize we are using critical portions of the Semantic Web (i.e. Data-Web) without even knowing it. Guess what? It's already happening. Simple case in point, you may have started to notice the emergence of SIOC gems in the same way you may have observed those RSS 2.0 gems at the dawn of Web 2.0. What I am implying here is that the real question we should be asking is: Where is the Semantic Web Data? And how easy or difficult will it be to generate? And where are the tools? My answers are presented below:

1. Pingthesemanticweb.com - Semantic Web Data Source Lookup & Tracking Service
2. Swoogle - Semantic Web Ontology Location Service
3. Semantic Web Solutions for Generating RDF Data from SQL Data
4. Semantic Web Solutions Directory
5. SIOC Project - Semantically-Interlinked Online Communities Ontology, a grassroots effort that provides a critical bridge between Web 2.0 and the Data-Web. For instance, existing Web 2.0 application profiles such as; Blogs, Wikis, Feed Aggregators, Content Managers, Discussion Forums etc.. are much closer to the Data-Web than you may think :-)
6. Virtuoso - our Universal Server for the Data-Web
7. OpenLink Data Spaces (ODS) - our SIOC based platform for transparent incorporation of the Data-Web into Web 1.0 and Web 2.0

Next stop, less writing, more demos, these are long overdue! At least from my side of the fence :-) I need to produce a little step-by-guide oriented screencasts that demonstrates how Web 2.0 meshes nicely with the Data-Web.

Here are some (not so end-user friendly) examples of how you can use SPARQL (Data-Web's Query Language) to query Web 2.0 Instance Data projected through the SIOC Ontology:

1. Weblog Data Query
2. Wiki Data Query
3. Aggregated Feeds Data Query - (RSS 1.0, RSS 2.0, Atom etc)
4. Shared Bookmarks Data Space
5. Web Filesystem Data Query - (Briefcase - Virtual Spotlight of sorts)
6. Photo Gallery Data Query (this could be data from Flickr etc..)
7. Discussion Data Query (e.g. Blog posts comments)
8. Data Queries across different Data Spaces - combining data from Wikis, Blogs, Feeds, Photos, Bookmarks, Discussions etc..

Note: You can use the online SPARQL Query Interface at: http://demo.openlinksw.com/isparql.

Other Data-Web Technology usage demos include:

1. TimBL's Tabulator - A Data-Web Browser
2. Semantic Web Client Library - RDF Data Drill Down Demos using SPARQL
3. Semantic Radar - A Firefox plug-in for auto-discovering SIOC Instance Data
4. Talk Digger - SIOC based Web Conversation Tracker

Amongst the numerous comments about this subject, I felt most compelled to respond to the commentary from Tim O'Reilly (based on his proximity to Web 2.0 etc..) in relation to his view that the NYT's Web 3.0 = Collective Intelligence Harnessing aspect of his Web 2.0 meme.

My response is dumped semi-verbatim below:

Tim,

A few things:

1. We are in an innovation continuum
2. The Web as a medium of innovation will evolve forever
3. Different commentators have different views about monikers associated with these innovations
4. To say Web 3.0 (aka the Data Web or Semantic Web - Layer 1) is what Web 2.0's collective intelligence is all about is a little inaccurate (IMHO); Web 2.0 doesn't provide "Open Data Access"
5. Web 2.0 is a "Web of Services" primarily, a dimension of "Web Interaction" defined by interaction with Services
6. Web 3.0 ("Data Web" or "Web of Databases" or "Semantic Web - Layer 1") is a Web dimension that provides "Open Data Access" that will be exemplified by the transition from "Mash-ups" (brute force data joining) to "Mesh-ups" (natural data joining)

The original "Web of Hypertext" or "Interactive Web", the current "Web of Services", and the emerging "Data Web" or "Web of Databases" collectively provide dimensions of interaction in the innovation continuum called the Web.

There are many more dimensions to come. Monikers come and go, but the retrospective "Long Shadow" of Innovation is ultimately timeless.

"Mutual Inclusivity" is a critical requirement for truly perceiving these "Web Interaction Dimensions" ("Participation" if I recall). "Mutual Exclusivity" on the other hand, simpy leads to obscuring reality with Versionitis as exemplified by the ongoing: Web 1.0 vs 2.0 vs 3.0 debates.

BTW - I enjoyed reading Nick Carr's take on the Web 3.0 meme, especially his "tongue in cheek" power-grab for the rights to all "Web 3.0" Conferences etc. :-)

Likewise, It would be nice if there were some Mash-ups, Service Endpoints, or Syndication Feeds that exposed relevant Data from the Web 2.0 Summit (beyond the usual selective, best-of, type Blog Commentary and traditional Speakers List).

From the ISWC event (#swig) I just located a presentation by TimBL titled: Semantic Web & Web 2.0

Key Excerpt: Web 2.0 and the Semantic Web work "Well Apart" and "Great Together".

What is this?

A declarative language adapted from SPARQL's graph pattern language (N3/Turtle) for mapping SQL Data to RDF Ontologies. We currently refer to this as a Graph Pattern based RDF VIEW Definition Language.

Why is it important?

It provides an effective mechanism for exposing existing SQL Data as virtual RDF Data Sets (Graphs) negating the data duplication associated with generating physical RDF Graphs from SQL Data en route to persistence in a dedicated Triple Store.

Enterprise applications (traditional and web based) and most Web Applications (Web 1.0 and Web 2.0) sit atop relational databases, implying that SQL/RDF model and data integration is an essential element of the burgeoning "Data Web" (Semantic Web - Layer 1) comprehension and adoption process.

In a nutshell, this is a quick route for non disruptive exposure of existing SQL Data to SPARQL supporting RDF Tools and Development Environments.

How does it work?

RDF Side

1. locate one or more Ontologies (e.g FOAF, SIOC, AtomOWL, SKOS etc.) that effectively defines the Concepts (Classes) and Terms (Predicates) to be exposed via your RDF Graph
2. Using the Virtuoso's RDF View Definition Language declare a International Resource Identifier (or URI) for your Graph. Example:

   CREATE GRAPH IRI("http://myopenlink.net/dataspace")

3. Then create Classes (Concepts), Class Properties/Predicates (Memb), and Class Instances (Inst) for the new Graph. Example:

   CREATE IRI CLASS odsWeblog:feed_iri  "http://myopenlink.net/dataspace/kidehen/weblog/MyFeeds" (
     in memb varchar not null, in inst varchar not null)

SQL Side

1. If Virtuoso isn't your SQL Data Store, Identify the ODBC or JDBC SQL data source(s) containing the SQL data to be mapped to RDF and then link the relevant tables into Virtuoso's Virtual DBMS Layer
2. Then use the RDF View Definition Language's graph pattern feature to generate SQL to RDF Mapping Template for your Graph. As shown in this ODS Weblog -> AtomOWL Mapping example.

Dimension 1 = Interactive Web (Visual Web of HTML based Sites aka Web 1.0)

Dimension 2 = Services Web (Presence based Web of Services; a usage pattern commonly referred to as Web 2.0)

Dimension 3 = Data Web (Presence and Open Data Access based Web of Databases aka Semantic Web layer 1)

Dimension 4 = Ontology Web (Intelligent Agent palatable Web aka Semantic Web layer 2)

....

Hopefully, I can expand further :-)

For additional clarity re. my comments above, you can also look at the SPARQL & SIOC Usecase samples document for our OpenLink Data Spaces platform. Bottom line, the Semantic Web and SPARQL aren't BORING. In fact, quite the contrary, since they are essential ingredients of a more powerful Web than the one we work with today!

Enjoy the rest of John's post:

Creating connections between discussion clouds with SIOC:

(Extract from our forthcoming BlogTalk paper about browsers for SIOC.)

SIOC provides a unified vocabulary for content and interaction description: a semantic layer that can co-exist with existing discussion platforms. Using SIOC, various linkages are created between the aforementioned concepts, which allow new methods of accessing this linked data, including:

• Virtual Forums. These may be a gathering of posts or threads which are distributed across discussion platforms, for example, where a user has found posts from a number of blogs that can be associated with a particular category of interest, or an agent identifies relevant posts across a certain timeframe.
• Distributed Conversations. Trackbacks are commonly used to link blog posts to previous posts on a related topic. By creating links in both directions, not only across blogs but across all types of internet discussions, conversations can be followed regardless of what point or URI fragment a browser enters at.
• Unified Communities. Apart from creating a web page with a number of relevant links to the blogs or forums or people involved in a particular community, there is no standard way to define what makes up an online community (apart from grouping the people who are members of that community using FOAF or OPML). SIOC allows one to simply define what objects are constituent parts of a community, or to say to what community an object belongs (using sioc:has_part / part_of): users, groups, forums, blogs, etc.
• Shared Topics. Technorati (a search engine for blogs) and BoardTracker (for bulletin boards) have been leveraging the free-text tags that people associate with their posts for some time now. SIOC allows the definition of such tags (using the subject property), but also enables hierarchial or non-hierarchial topic definition of posts using sioc:topic when a topic is ambiguous or more information on a topic is required. Combining with other Semantic Web vocabularies, tags and topics can be further described using the SKOS organisation system.
• One Person, Many User Accounts. SIOC also aims to help the issue of multiple identities by allowing users to define that they hold other accounts or that their accounts belong to a particular personal identity (via foaf:holdsOnlineAccount or sioc:account_of). Therefore, all the posts or comments made by a particular person using their various associated user accounts across platforms could be identified.

A phase in the evolution web usage patterns that emphasizes Web Services based interaction between “Web Users” and “Points of Web Presence” over traditional “Web Users” and “Web Sites” based interaction. Basically, a transition from visual site interaction to presence based interaction.

BTW - Dare Obasanjo also commented about Web usage patterns in his post titled: The Two Webs. Where he concluded that we had a dichotomy along the lines of: HTTP-for-APIs (2.0) and HTTP-for-Browsers (1.0). Which Jon Udell evolved into: HTTP-Services-Web and HTTP-Intereactive-Web during our recent podcast conversation.

With definitions in place, I will resume my quest to unveil the aforementioned Web 2.0 Data Access Conundrum:

• Emphasis on XML's prowess in the realms of Data and Protocol Modeling alongside Data Representation. Especially as SOAP or REST styles of Web Services and various XML formats (RSS 0.92/1.0/1.1/2.0, Atom, OPML, OCS etc.) collectively define the Web 2.0 infrastructure landscape
• Where a modicum of Data Access appreciation and comprehension does exist it is inherently compromised by business models that mandate some form of “Walled Gardens” and “Data Silos”
• Mash-ups are a response to said “Walled Gardens” and “Data Silos” . Mash-ups by definition imply combining things that were not built for recombination.

As you can see from the above, Open Data access isn't genuinely compatible with Web 2.0.

We can also look at the same issue by way of the popular M-V-C (Model View Controller) pattern. Web 2.0 is all about the “V” and “C” with a modicum of “M” at best (data access, open data access, and flexible open data access are completely separate things). The “C” items represent application logic exposed by SOAP or REST style web services etc. I'll return to this later in this post.

What about Social Networking you must be thinking? Isn't this a Web 2.0 manifestation? Not at all (IMHO). The Web was developed / invented by Tim Berners-Lee to leverage the “Network Effects” potential of the Internet for connecting People and Data. Social Networking on the other hand, is simply one of several ways by which construct network connections. I am sure we all accept the fact that connections are built for many other reasons beyond social interaction. That said, we also know that through social interactions we actually develop some of our most valuable relationships (we are social creatures after-all).

The Web 2.0 Open Data Access impedance reality is ultimately going to be the greatest piece of tutorial and usecase material for the Semantic Web. I take this position because it is human nature to seek Freedom (in unadulterated form) which implies the following:

• Access Data from a myriad of data sources (irrespective of structural differences at the database level)
• Mesh (not Mash) data in new and interesting ways
• Share the meshed data with as many relevant people as possible for social, professional, political, religious, and other reasons
• Construct valuable networks based on data oriented connections

Web 2.0 by definition and use case scenarios is inherently incompatible with the above due to the lack of Flexible and Open Data Access.

If we take the definition of Web 2.0 (above) and rework it with an appreciation Flexible and Open Data Access you would arrive at something like this:

A phase in the evolution of the web that emphasizes interaction between “Web Users” and “Web Data” facilitated by Web Services based APIs and an Open & Flexible Data Access Model “.

In more succinct form:

A pervasive network of people connected by data or data connected by people.

Returning to M-V-C and looking at the definition above, you now have a complete of ”M“ which is enigmatic in Web 2.0 and the essence of the Semantic Web (Data and Context).

To make all of this possible a palatable Data Model is required. The model of choice is the Graph based RDF Data Model - not to be mistaken for the RDF/XML serialization which is just that, a data serialization that conforms to the aforementioned RDF data model.

The Enterprise Challenge

Web 2.0 cannot and will not make valuable inroads into the the enterprise because enterprises live and die by their ability to exploit data. Weblogs, Wikis, Shared Bookmarking Systems, and other Web 2.0 distributed collaborative applications profiles are only valuable if the data is available to the enterprise for meshing (not mashing).

A good example of how enterprises will exploit data by leveraging networks of people and data (social networks in this case) is shown in this nice presentation by Accenture's Institute for High Performance Business titled: Visualizing Organizational Change.

Web 2.0 commentators (for the most part) continue to ponder the use of Web 2.0 within the enterprise while forgetting the congruency between enterprise agility and exploitation of people & data networks (The very issue emphasized in this original Web vision document by Tim Berners-Lee). Even worse, they remain challenged or spooked by the Semantic Web vision because they do not understand that Web 2.0 is fundamentally a Semantic Web precursor due to Open Data Access challenges. Web 2.0 is one of the greatest demonstrations of why we need the Semantic Web at the current time.

Finally, juxtapose the items below and you may even get a clearer view of what I am an attempting to convey about the virtues of Open Data Access and the inflective role it plays as we move beyond Web 2.0:

Information Management Proposal - Tim Berners-Lee
Visualizing Organizational Change - Accenture Institute of High Performance Business

This is fundamentally an animation demonstrating Semantic Web exploitation in the classic: picture speaks a thousand words manner. It also illustrates (yet again) the important Data Space(s) aspect of creating Semantic Web presence.

Finally, the Web 2.0 usage pattern tries to espouse what's demonstrated in this animation via data-context-challenged interactions (due to its "Walled Garden" and "Data Silo" approach to Data Access etc..). The Semantic Web (as per numerous posts on the subject) on the other hand achieves this via data-context-aware interactions (as will be exemplified via meshups).

One of the great things about the moderate “open data access” that we have today (courtesy of the blogosphere) is the fact that you can observe the crystallization of new thinking, and/or new appreciation of emerging ideas, in near real-time. Of course, when we really hit the tracks with the Semantic Web this will be in “conditional real-time” (i.e. you choose and control your scope and sensitivity to data changes etc..).

For instance, by way of feed subscriptions, I stumbled upon a series of posts by Jason Kolb that basically articulate what I (and others who believe in the Semantic Web vision) have been attempting to convey in a myriad of ways via posts and commentary etc..

Here are the links to the 4 part series by Jason:

1. Reinventing the Internet part 1 (appreciating “Presence” over traditional “Web Sites”)
2. Reinventing the Internet part 2
3. Reinventing the Internet part 3 (appreciating and comprehending URIs)
4. Reinventing the Internet part 4 (nice visualization of what “Data Spaces”)
5. Reinventing the Internet part 5 (everyone will have a Data Space in due course becuase the Internet is really a Federation of Data Spaces)
   

A phase in the evolution web usage patterns that emphasizes Web Services based interaction between “Web Users” and “Points of Web Presence” over traditional “Web Users” and “Web Sites” based interaction. Basically, a transition from visual site interaction to presence based interaction.

BTW - Dare Obasanjo also commented about Web usage patterns in his post titled: The Two Webs. Where he concluded that we had a dichotomy along the lines of: HTTP-for-APIs (2.0) and HTTP-for-Browsers (1.0). Which Jon Udell evolved into: HTTP-Services-Web and HTTP-Intereactive-Web during our recent podcast conversation.

With definitions in place, I will resume my quest to unveil the aforementioned Web 2.0 Data Access Conundrum:

• Emphasis on XML's prowess in the realms of Data and Protocol Modeling alongside Data Representation. Especially as SOAP or REST styles of Web Services and various XML formats (RSS 0.92/1.0/1.1/2.0, Atom, OPML, OCS etc.) collectively define the Web 2.0 infrastructure landscape
• Where a modicum of Data Access appreciation and comprehension does exist it is inherently compromised by business models that mandate some form of “Walled Gardens” and “Data Silos”
• Mash-ups are a response to said “Walled Gardens” and “Data Silos” . Mash-ups by definition imply combining things that were not built for recombination.

As you can see from the above, Open Data access isn't genuinely compatible with Web 2.0.

We can also look at the same issue by way of the popular M-V-C (Model View Controller) pattern. Web 2.0 is all about the “V” and “C” with a modicum of “M” at best (data access, open data access, and flexible open data access are completely separate things). The “C” items represent application logic exposed by SOAP or REST style web services etc. I'll return to this later in this post.

What about Social Networking you must be thinking? Isn't this a Web 2.0 manifestation? Not at all (IMHO). The Web was developed / invented by Tim Berners-Lee to leverage the “Network Effects” potential of the Internet for connecting People and Data. Social Networking on the other hand, is simply one of several ways by which construct network connections. I am sure we all accept the fact that connections are built for many other reasons beyond social interaction. That said, we also know that through social interactions we actually develop some of our most valuable relationships (we are social creatures after-all).

The Web 2.0 Open Data Access impedance reality is ultimately going to be the greatest piece of tutorial and usecase material for the Semantic Web. I take this position because it is human nature to seek Freedom (in unadulterated form) which implies the following:

• Access Data from a myriad of data sources (irrespective of structural differences at the database level)
• Mesh (not Mash) data in new and interesting ways
• Share the meshed data with as many relevant people as possible for social, professional, political, religious, and other reasons
• Construct valuable networks based on data oriented connections

Web 2.0 by definition and use case scenarios is inherently incompatible with the above due to the lack of Flexible and Open Data Access.

If we take the definition of Web 2.0 (above) and rework it with an appreciation Flexible and Open Data Access you would arrive at something like this:

A phase in the evolution of the web that emphasizes interaction between “Web Users” and “Web Data” facilitated by Web Services based APIs and an Open & Flexible Data Access Model “.

In more succinct form:

A pervasive network of people connected by data or data connected by people.

Returning to M-V-C and looking at the definition above, you now have a complete of ”M“ which is enigmatic in Web 2.0 and the essence of the Semantic Web (Data and Context).

To make all of this possible a palatable Data Model is required. The model of choice is the Graph based RDF Data Model - not to be mistaken for the RDF/XML serialization which is just that, a data serialization that conforms to the aforementioned RDF data model.

The Enterprise Challenge

Web 2.0 cannot and will not make valuable inroads into the the enterprise because enterprises live and die by their ability to exploit data. Weblogs, Wikis, Shared Bookmarking Systems, and other Web 2.0 distributed collaborative applications profiles are only valuable if the data is available to the enterprise for meshing (not mashing).

A good example of how enterprises will exploit data by leveraging networks of people and data (social networks in this case) is shown in this nice presentation by Accenture's Institute for High Performance Business titled: Visualizing Organizational Change.

Web 2.0 commentators (for the most part) continue to ponder the use of Web 2.0 within the enterprise while forgetting the congruency between enterprise agility and exploitation of people & data networks (The very issue emphasized in this original Web vision document by Tim Berners-Lee). Even worse, they remain challenged or spooked by the Semantic Web vision because they do not understand that Web 2.0 is fundamentally a Semantic Web precursor due to Open Data Access challenges. Web 2.0 is one of the greatest demonstrations of why we need the Semantic Web at the current time.

Finally, juxtapose the items below and you may even get a clearer view of what I am an attempting to convey about the virtues of Open Data Access and the inflective role it plays as we move beyond Web 2.0:

Information Management Proposal - Tim Berners-Lee
Visualizing Organizational Change - Accenture Institute of High Performance Business

Continuing from our recent Podcast conversation, Jon Udell sheds further insight into the essence of our conversation via a “Strategic Developer” column article titled: Accessing the web of databases.

Below, I present an initial dump of a DataSpace FAQ below that hopefully sheds light on the DataSpace vision espoused during my podcast conversation with Jon.

What is a DataSpace?

A moniker for Web-accessible atomic containers that manage and expose Data, Information, Services, Processes, and Knowledge.

What would you typically find in a Data Space? Examples include:

• Raw Data - SQL, HTML, XML (raw), XHTML, RDF etc.
  
  
• Information (Data In Context) - XHTML (various microformats), Blog Posts (in RSS, Atom, RSS-RDF formats), Subscription Lists (OPML, OCS, etc), Social Networks (FOAF, XFN etc.), and many other forms of applied XML.

• Web Services (Application/Service Logic) - REST or SOAP based invocation of application logic for context sensitive and controlled data access and manipulation.

• Persisted Knowledge - Information in actionable context that is also available in transient or persistent forms expressed using a Graph Data Model. A modern knowledgebase would more than likely have RDF as its Data Language, RDFS as its Schema Language, and OWL as its Domain  Definition (Ontology) Language. Actual Domain, Schema, and Instance Data would be serialized using formats such as RDF-XML, N3, Turtle etc).

How do Data Spaces and Databases differ?
Data Spaces are fundamentally problem-domain-specific database applications. They offer functionality that you would instinctively expect of a database (e.g. AICD data management) with the additonal benefit of being data model and query language agnostic. Data Spaces are for the most part DBMS Engine and Data Access Middleware hybrids in the sense that ownership and control of data is inherently loosely-coupled.

How do Data Spaces and Content Management Systems differ?
Data Spaces are inherently more flexible, they support multiple data models and data representation formats. Content management systems do not possess the same degree of data model and data representation dexterity.

How do Data Spaces and Knowledgebases differ?
A Data Space cannot dictate the perception of its content. For instance, what I may consider as knowledge relative to my Data Space may not be the case to a remote client that interacts with it from a distance, Thus, defining my Data Space as Knowledgebase, purely, introduces constraints that reduce its broader effectiveness to third party clients (applications, services, users etc..). A Knowledgebase is based on a Graph Data Model resulting in significant impedance for clients that are built around alternative models. To reiterate, Data Spaces support multiple data models.

What Architectural Components make up a Data Space?

• ORDBMS Engine - for Data Modeling agility (via complex purpose specific data types and data access methods), Data Atomicity, Data Concurrency, Transaction Isolation, and Durability (aka ACID).
  
  
• Virtual Database Engine - for creating a single view of, and access point to, heterogeneous SQL, XML, Free Text, and other data. This is all about Virtualization at the Data Access Level.

• Web Services Platform - enabling controlled access and manipulation (via application, service, or protocol logic) of Virtualized or Disparate Data. This layer handles the decoupling of functionality from monolithic wholes for function specific invocation via Web Services using either the SOAP or REST approach.

Where can I see a DataSpace along the lines described, in action?

Just look at my blog, and take the journey as follows:

• Front Door (Web 1.0)
• Lounge (Web 2.0) via GData or OpenSearch
• Floor Plan via FOAF or SIOC RDF Data Sets (Graphs)
• Rest of the house (beyond Web 2.0) sending  SPARQL Queries to a SPARQL Endpoint.
  

What about other Data Spaces?

There are several and I will attempt to categorize along the lines of query method available:
Type 1 (Free Text Search over HTTP):
Google, MSN, Yahoo!, Amazon, eBay, and most Web 2.0 plays .

Type 2 (Free Text Search and XQuery/XPath over HTTP)
A few blogs and Wikis (Jon Udell's and a few others)

•    SIOC enabled sites (aka points of semantic web presence)
  
•    PingTheSemantic
  

• My Blog Data Space (as stated earlier in this post)
  
• My General Data Space - (ditto; note that this is currently experimental)
  

What About Data Space aware tools?

•    OpenLink Ajax Toolkit - provides Javascript Control level binding to Query Services such as XMLA for SQL, GData for Free Text, OpenSearch for Free Text, SPARQL for RDF, in addition to service specific Web Services (Web 2.0 hosted solutions that expose service specific APIs)
•    Semantic Radar - a Firefox Extension
•    PingTheSemantic - the Semantic Webs equivalent of Web 2.0's weblogs.com
•    PiggyBank - a Firefox Extension

BTW - I am just as perplexed as Dare about Paul Graham being blind-sided by the integration of Calendaring and Email by Google.

Paul Graham was Surprised by Google Calendar?: "

I was just reading Paul Graham's post entitled The Kiko Affair which talks about the recent failure of Kiko, an AJAX web-calendaring application. I was quite surprised to see the following sentence in Paul Graham's post

The killer, unforseen by the Kikos and by us, was Google Calendar's integration with Gmail. The Kikos can't very well write their own Gmail to compete.

Integrating a calendaring application with an email application seems pretty obvious to me especially since the most popular usage of calendaring applications is using Outlook/Exchange to schedule meetings in corporate environments. What's surprising to me is how surprised people are that an idea that failed in 1990s will turn out any differently now because you sprinkle the AJAX magic pixie dust on it.

Kiko was a feature, not a full-fledged online destination let alone a viable business. There'll be a lot more entrants into the TechCrunch deadpool that are features masquerading as companies before the 'Web 2.0' hype cycle runs its course.

"

(Via Dare Obasanjo aka Carnage4Life.)

OAT offers a broad Javascript-based, browser-independent widget set
for building data source independent rich internet applications that are usable across a broad range of Ajax-capable web browsers.

OAT's support binding to the following data sources via its Ajax Database Connectivity Layer:

SQL Data via XML for Analysis (XMLA)
Web Data via SPARQL, GData, and OpenSearch Query Services
Web Services specific Data via service specific binding to SOAP and REST style web services

The toolkit includes a collection of powerful rich internet application prototypes include: SQL Query By Example, Visual Database Modeling, and Data bound Web Form Designer.

Project homepage on sourceforge.net:

http://sourceforge.net/projects/oat

Source Code:

http://sourceforge.net/projects/oat/files

Live demonstration:

http://www.openlinksw.com/oat/

Goggle vs Semantic Web: "Google exec challenges Berners-Lee 'At the end of the keynote, however, things took a different turn. Google Director of Search and AAAI Fellow Peter Norvig was the first to the microphone during the Q&A session, and he took the opportunity to raise a few points.

'What I get a lot is: 'Why are you against the Semantic Web?' I am not against the Semantic Web. But from Google's point of view, there are a few things you need to overcome, incompetence being the first,' Norvig said. Norvig clarified that it was not Berners-Lee or his group that he was referring to as incompetent, but the general user.'

Related: Google Base -- summing up."

(Via More News.)

When will we drop the ill conceived notion that end-users are incompetent?

Has it every occurred to software developers and technology vendors that incompetent, dumb, and other contemptuous end-user adjectives simply reflect the inability of most technology products to surmount end-user "Interest Activation Thresholds"?

Interest Activation Threshold (IAT)? What's That?

I have a fundamental personal belief that all human beings are intelligent. Our ability to demonstrate intelligence, or be perceived as intelligent, is directly proportional to our interest level in a given context. In short, we have "Ambivalence Quotients" (AQs) just as we have "Intelligence Quotients" (IQs).

An interested human being is an inherently intelligent entity. The abstract nature of human intelligence also makes locating the IQ and AQ on/off buttons a mercurial quest at the best of times.

Technology end-users exhibit high AQs, most of the time due to the inability of most technology products to truly engage, and ultimately stimulate genuine interest, by surmounting IAT and reducing AQ.

Ironically, when a technology vendor is lagging behind its competitors in the "features arms race" it is common place to use the familiar excuse: "our end-users aren't asking for this feature".

Note To Google:

Ambivalence isn't incompetence. If end-users were genuinely incompetent, how is that they run rings around your page rank algorithms by producing google-friendly content at the expense of valuable context? What about the deteriorating value of Adsense due to click fraud? Likewise, the continued erosion of the value of your once exemplary "keyword based search" service? As we all know, necessity is the mother of invention, so when users develop high AQs because there is nothing better, we end up with a forced breech of "IAT"; which is why the issues that I mention remain long term challenges for you. Ironically, the so called "incompetents" are already outsmarting you, and you don't seem to comprehend this reality or its inevitable consequences.

Finally, how you are going to improve value without integrating the Semantic Web vision into your R&D roadmap? I can tell you categorically that you have little or no wiggle room re. this matter, especially if you want to remain true to your: "don't be evil" mantra. My guess is that you will incorporate Semantic Web technologies sooner rather than later (Google Co-op is a big clue). I would even go as far as predicting a Google hosted SPARQL Query Endpoint alongside your GData endpints during the next 6-12 months (if even that long). I believe that your GData protocol (like the rest of Web 2.0) will ultimately accelerate your appreciation of the data model dexterity that RDF brings to loosely coupled knowledge networks espoused by the Semantic Web vision.

Google & Semantic Web Paradox

The Semantic Web vision has the RDF graph data model at its core (and for good reason), but even more confusing for me, as I process Google sentiments about the Semantic Web, is the fact that RDF's actual creator (Ramanathan Guha aka. Guha) currently works at Google. There's a strange disconnect here IMHO.

If I recall correctly, Google wants to organize the worlds data and information, leaving the knowledge organization to someone else which is absolutely fine. What is increasingly irksome, is the current tendency to use corporate stature to generate Fear, Uncertainty, and Doubt when the subject matter is the "Semantic Web".

BTW - I've just read Frederick Giasson's perspective on the Google Semantic Web paradox which ultimately leads to the same conclusions regarding Google's FUD stance when dealing with matters relating to the Semantic Web.

I wonder if anyone is tracking the google hits for "fud google semantic web"?

The other day I ran into Jitsu, a new toolkit for creating Ajax-y applications.

Jitsu takes an interesting approach in the Ajaxy space."

(Via Miguel de Icaza.)

Web 2.0 Self-Experiment: "

I shopped for everything except food on eBay. When working with foreign-language documents, I used translations from Babel Fish. (This worked only so well. After a Babel Fish round-trip through Italian, the preceding sentence reads, 'That one has only worked therefore well.') Why use up space storing files on my own hard drive when, thanks to certain free utilities, I can store them on Gmail's servers? I saved, sorted, and browsed photos I uploaded to Flickr. I used Skype for my phone calls, decided on books using Amazon's recommendations rather than 'expert' reviews, killed time with videos at YouTube, and listened to music through customizable sites like Pandora and Musicmatch. I kept my schedule on Google Calendar, my to-do list on Voo2do, and my outlines on iOutliner. I voyeured my neighborhood's home values via Zillow. I even used an online service for each stage of the production of this article, culminating in my typing right now in Writely rather than Word. (Being only so confident that Writely wouldn't somehow lose my work -- or as Babel Fish might put it, 'only confident therefore' -- I backed it up into Gmail files.

(Via Valentin Zacharias (Student).)

Tim O'Reilly's response provides the following hierarchy for Web 2.0 based on The what he calls: "Web 2.0-ness":

level 3: The application could ONLY exist on the net, and draws its essential power from the network and the connections it makes possible between people or applications. These are applications that harness network effects to get better the more people use them. EBay, craigslist, Wikipedia, del.icio.us, Skype, (and yes, Dodgeball) meet this test. They are fundamentally driven by shared online activity. The web itself has this character, which Google and other search engines have then leveraged. (You can search on the desktop, but without link activity, many of the techniques that make web search work so well are not available to you.) Web crawling is one of the fundamental Web 2.0 activities, and search applications like Adsense for Content also clearly have Web 2.0 at their heart. I had a conversation with Eric Schmidt, the CEO of Google, the other day, and he summed up his philosophy and strategy as "Don't fight the internet." In the hierarchy of web 2.0 applications, the highest level is to embrace the network, to understand what creates network effects, and then to harness them in everything you do.

Level 2: The application could exist offline, but it is uniquely advantaged by being online. Flickr is a great example. You can have a local photo management application (like iPhoto) but the application gains remarkable power by leveraging an online community. In fact, the shared photo database, the online community, and the artifacts it creates (like the tag database) is central to what distinguishes Flickr from its offline counterparts. And its fuller embrace of the internet (for example, that the default state of uploaded photos is "public") is what distinguishes it from its online predecessors.

Level 1: The application can and does exist successfully offline, but it gains additional features by being online. Writely is a great example. If you want to do collaborative editing, its online component is terrific, but if you want to write alone, as Fallows did, it gives you little benefit (other than availability from computers other than your own.)

Level 0: The application has primarily taken hold online, but it would work just as well offline if you had all the data in a local cache. MapQuest, Yahoo! Local, and Google Maps are all in this category (but mashups like housingmaps.com are at Level 3.) To the extent that online mapping applications harness user contributions, they jump to Level 2.

So, in a sense we have near conclusive confirmation that Web 2.0 is simply about APIs (typically service specific Data Silos or Walled-gardens) with little concern, understanding, or interest in truly open data access across the burgeoning "Web of Databases". Or the Web of "Databases and Programs" that I prefer to describe as "Data Spaces"

Thus, we can truly begin to conclude that Web 3.0 (Data Web) is the addition of Flexible and Open Data Access to Web 2.0; where the Open Data Access is achieved by leveraging Semantic Web deliverables such as the RDF Data Model and the SPARQL Query Language :-)

Standards as social contracts: "Looking at Dave Winer's efforts in evangelizing OPML, I try to draw some rough lines into what makes a de-facto standard. De Facto standards are made and seldom happen on their own. In this entry, I look back at the history of HTML, RSS, the open source movement and try to draw some lines as to what makes a standard.

(Via Tristan Louis.)

I posted a comment to the Tristan Louis' post along the following lines:

Analysis is spot on re. the link between de facto standardization and bootstrapping. Likewise, the clear linkage between boostrapping and connected communities (a variation of the social networking paradigm).

Dave built a community around a XML content syndication and subscription usecase demo that we know today as the blogosphere. Superficially, one may conclude that Semantic Web vision has suffered to date from a lack a similar bootstrap effort. Whereas in reality, we are dealing with "time and context" issues that are critical to the base understanding upon which a "Dave Winer" style bootstrap for the Semantic Web would occur.

Personally, I see the emergence of Web 2.0 (esp. the mashups phenomenon) as the "time and context" seeds from which the Semantic Web bootstrap will sprout. I see shared ontologies such as FOAF and SIOC leading the way (they are the RSS 2.0's of the Semantic Web IMHO).

This particular inflection and, ultimately, transistion is going to occur at Warp Speed!

"We all know that structured data is boring and useless; while unstructured data is sexy and chock full of value. Well, only up to a point, Lord Copper. Genuinely unstructured data can be a real nuisance - imagine extracting the return address from an unstructured letter, without letterhead and any of the formatting usually applied to letters. A letter may be thought of as unstructured data, but most business letters are, in fact, highly-structured." ....

"The labels "structured data" and "unstructured data" are often used ambiguously by different interest groups; and often used lazily to cover multiple distinct aspects of the issue. In reality, there are at least three orthogonal aspects to structure:

* The structure of the data itself.

* The structure of the container that hosts the data.

* The structure of the access method used to access the data.

These three dimensions are largely independent and one does not need to imply another. For example, it is absolutely feasible and reasonable to store unstructured data in a structured database container and access it by unstructured search mechanisms."

Data understanding and appreciation is dwindling at a time when the reverse should be happening. We are supposed to be in the throws of the "Information Age", but for some reason this appears to have no correlation with data and "data access" in the minds of many -- as reflected in the broad contradictory positions taken re. unstructured data vs structured data, structured is boring and useless while unstructured is useful and sexy....

The difference between "Structured Containers" and "Structured Data" are clearly misunderstood by most (an unfortunate fact).

For instance all DBMS products are "Structured Containers" aligned to one or more data models (typically one). These products have been limited by proprietary data access APIs and underlying data model specificity when used in the "Open-world" model that is at the core of the World Wide Web. This confusion also carries over to the misconception that Web 2.0 and the Semantic/Data Web are mutually exclusive.

But things are changing fast, and the concept of multi-model DBMS products is beginning to crystalize. On our part, we have finally released the long promised "OpenLink Data Spaces" application layer that has been developed using our Virtuoso Universal Server. We have structured unified storage containment exposed to the data web cloud via endpoints for querying or accessing data using a variety of mechanisms that include; GData, OpenSearch, SPARQL, XQuery/XPath, SQL etc..

To be continued....

The screencasts covered the following functionality realms:

1. SQL Query By Example (basic)
2. SQL Query By Example (advanced - pivot table construction)
3. Web Form Design (basic database driven map based mashup)
4. Web Form Design (advanced database driven map based mashup)

To bring additional clarity to the screencasts demos and OAT in general, I have saved a number of documents that are the by products of activities in the screenvcasts:

1. Live XML Document produced using SQL Query By Example (basic) (you can use drag and drop columns across the grid to reorder and sort presentation)
2. Live XML Document produced using QBE and Pivot Functionality (you can drag and drop the aggregate columns and rows to create your own views etc..)
3. Basic database driven map based mashup (works with FireFox, Webkit, Camino; click on pins to see national flag)
4. Advanced database driven map based mashup (works with FireFox, Webkit, Camino; records, 36, 87, and 257 will unveil pivots via lookup pin)

Notes:

• “Advanced”, as used above, simply means that I am embedding images (employee photos and national flags) and a database driven pivot into the map pins that serve as details lookups in classic SQL master/details type scenarios.
• The “Ajax Call In Progress..” dialog is there to show live interaction with a remote database (in this case Virtuoso but this could be any ODBC, JDBC, OLEDB, ADO.NET, or XMLA accessible data source)
• The data access magic source (if you want to call it that) is XMLA - a standard that has been in place for years but completely misunderstood and as a result under utilized

You can see a full collection of saved documents at the following locations:

• My Mashups demo directory (Google and Yahoo! demo variants but note these do not work with Safari or IE at the current time. IE7 issues will be resolved in the next day or so)
• My Pivots demo directory (other Pivots will be added as I build and save them)
• My Saved Queries (a collection of saved QBE generated queries)

Despite page ranking and other techniques, the scale of the Internet is straining available commercial search engines to deliver truly relevant content.' This observation is not new, but its relevance is growing.' Similarly, the integration and interoperabillity challenges facing enterprises have never been greater.' One approach to address these needs, among others, is to adopt semantic Web standards and technologies.

The image is compelling:' targeted and unambiguous information from all relevant sources, served in usable bit-sized chunks.' It sounds great; why isn’t it happening?

There are clues — actually, reasons — why semantic Web technology is not being embraced on a broad-scale way.' I have spoken elsewhere as to why enterprises or specific organizations will be the initial adopters and promoters of these technologies.' I still believe that to be the case.' The complexity and lack of a network effect ensure that semantic Web stuff will not initially arise from the public Internet.

Parellels with Knowledge Management

Paul Warren, in' '

(Via AI3 - Adaptive Information:::.)

A pre-print from Tim Finin and Li Deng entitled, Search Engines for Semantic Web Knowledge,¹ presents a thoughtful and experienced overview of the challenges posed to conventional search by semantic Web constructs.' The authors’ base much of their observations on their experience with the Swoogle semantic Web search engine over the past two years.' They also used Swoogle, whose index contains information on over 1.3M RDF documents, to generate statistics on the semantic Web size and growth in the paper.

Among other points, the authors note these key differences and challenges from conventional search engines:

• Harvesting — the need to discriminantly discover semantic Web documents and to accurately index their semi-structured components
• Search - the need for search to cover a broader range than documents in a repository, going from the universal to the atomic granularity of a triple.' Path tracing and provenance of the information may also be important
• Rank — results ranking needs to account for the contribution of the semi-structured data, and
  
• Archive — more versioning and tracking is needed since undelrying ontologies will surely grow and evolve.

The authors particularly note the challenge of indexing as repositories grow to actual Internet scales.

Though not noted, I would add to this list the challenge of user interfaces. Only a small percentage of users, for example, use Google’s more complicated advanced search form.' In its full-blown implementation, semantic Web search variations could make the advanced Google form look like child’s play.

'

¹Tim Finin and Li Ding, 'Search Engines for Semantic Web Knowledge,' a pre-print to be published in the Proceedings of XTech 2006: Building Web 2.0, May 16, 2006, 19 pp.' A PDF of the paper is available for download.

(Via AI3 - Adaptive Information:::.)

Two graphs that explain most IT dysfunction (Part I): "

Inspired by reading about other people’s blogging weaknesses, I’ve decided to finally get this one off the back burner and post it. I’m pretty sure that this isn’t original, but I started thinking about this way back in 1996 (pre-social-bookmarking) and I’ve lost my pointer to whatever influenced it. Anybody who can set me straight- I’d appreciate it.

So here goes.

There are two graphs which, when seen together, explain a hell of a lot about various forms of dysfunction that you see in the technology world.

In this first graph, X represents relative ‘technical expertise’ and Y represents the ‘perceived benefit’ in the introduction of a new technology:

The summary is that technical neophytes (A) tend to see high potential benefit in new technologies, while people who have a bit of technology experience (B) grow increasingly cynical about technology claims and can rattle-off the names of technologies that they have seen over-hyped and that have under-delivered. The interesting thing though, is that, as people become really expert in technology (C), their view of the potential benefits in new technology starts to increase again. At the far right of this scale I’m talking about the real experts- the alpha-geeks of the world.

In the second graph, X again represents technical expertise, but Y represents ‘perceived risk’ associated with the introduction of a new technology:

Here the curve is inverted, but the basic pattern is the same. The neophytes (A) are blissfully unaware of the things that can go wrong with the introduction of a new technology. The tech-savvy (B) are battle-scarred and have seen (and possibly caused) countless disasters. The alpha-geeks (C) have also seen their share of problems, but they have also learned from their mistakes and know how to avoid them in the future. The alpha-geeks understand how to manage the risk.

Now things get interesting when you map these two dynamics against each other:

You see that neophytes in group A have essentially the same world view as the alpha-geeks in group C, but for completely different reasons. The trouble starts when you realize that most of senior executives, venture capitalists and members of the popular press are in group A. At the other extreme, most R&D groups, architecture groups, independent consultancies, technology pundits, etc. are in group C . There are a few problems with this:

• People in group A will often talk to and solicit advice from people in group C
• There are relatively few people in group C
• Most of the people who actually have to implement new technologies are in group B.

So you can start to see the problem.

In Part II I’l talk some more about group B and I’ll discuss some of the classic patterns that emerge when A, B and C try to work with each other.

"

My too-long absence from writing much here can be ascribed to two, differently pleasant, activities. First, a fantastic vacation in Cuba, and second, the redesign and launch of the XTech web site.

Of the first, come to my place for dinner and I'll bore you at length about how amazing it was. Of the second, I'd like to bore you right now!

Thanks to Ruby on Rails and a few late nights, the XTech site now has these new features:

• Full conference schedule (apart from 6 Mozilla talks I'm still nailing down)
• A blog. With go-faster Atom 1.0 stripes and everything!
• Details on the newly-added Ajax Developers' Day.

A few more details on the Ajax Developers' Day. As I mentioned before, when putting together the schedule we felt there was a lot of excellent content still missed out (I'm still feeling guilty at having rejected proposals from many good friends and excellent speakers). So, we put together an extra day at the beginning of the conference where we could go further into detail on Ajax technologies.

This day, featuring speakers such as Simon Willison from Yahoo!, XML expert Kurt Cagle and OpenLaszlo's Max Carlson, will allow those working on Ajax projects--either deployment or toolkits--to meet, discuss best practice and move forward on new ideas. Although it's a day-long event, we didn't want to make the price tag as high as a full-day tutorial, so you can register for the cost of a half-day tutorial.

A few implementation details

If that all sounded a little like advertising, here are some technical details worth sharing. The site's CMS is built on Ruby on Rails. Development was done on Linux, with the help of WINE to check out the view from Internet Explorer. The newsletter is managed by the absurdly wonderful CampaignMonitor.

This conference not for sale

Before I went on vacation, there was some debate in various quarters about paid-for plenary and keynote slots in conferences. Though I hope it is obvious, I wanted to state where I, and thus the XTech conference, stand on this issue.

It has always been my policy to maintain a strict separation between the commercial and editorial aspects of XTech. Although each year there's always a company who thinks they can buy a speaking slot, I never let this happen. The content of the conference is formed by editorial selection by the programme committee, who take the scores from the peer review panel as their primary guide.

Aside from what I hope shows in the excellent quality of the talks and generally interesting keynotes (yes, we get it wrong occasionally!), there are two effects on the conference.

• Sponsors are that much more respected. When a sponsor respects the delegates' time and intelligence, but still attends, you know they're serious about engagement with attendees.
  
• A higher portion of the conference cost is in the registration fees than for some other conferences. We're still trying to keep the costs as low as we can, but we're not prepared to compromise the quality of the schedule by letting vendors buy talk time.

(Via Edd Dumbill's Weblog: Behind the Times.)

These are my notes from the session eBay Web Services: A Marketplace Platform for Fun and Profit by Adam Trachtenberg.

This session was about the eBay developer program. The talk started by going over the business models for 'Web 2.0' startups. Adam Trachtenberg surmised that so far only two viable models have shown up (i) get bought by Yahoo! and (ii) put a lot of Google AdSense ads on your site. The purpose of the talk was to introduce a third option, making money by integrating with eBay's APIs.

Adam Trachtenberg went on to talk about the differences between providing information and providing services. Information is read-only while services are read/write. Services have value because they encourage an 'architecture of participation'.

eBay is a global, online marketplace that facilitates the exchange of goods. The site started off as being a place to purchase used collectibles but now has grown to encompass old and new items, auctions and fixed price sales (fixed price sales are now a third of their sales) and even sales of used cars. There are currently 78 million items being listed at any given time on eBay.

As eBay has grown more popular they have come to realize that one size doesn't fit all when it comes to the website. It has to be customized to support different languages and markets as well as running on devices other the PC. Additionally, they discovered that some companies had started screen scraping their site to give an optimized user experience for some power users. Given how fragile screen scraping is the eBay team decided to provide a SOAP API that would be more stable and performant for them than having people screen scrape the website.

The API has grown to over 100 methods and about 43% of the items on the website are added via the SOAP API. The API enables one to build user experiences for eBay outside the web browser such as integration with cell phones, Microsoft Office, gadgets & widgets, etc. The API has an affiliate program so developers can make money for purchases that happen through the API. An example of the kind of mashup one can build to make money from the eBay API is https://www.dudewheresmyusedcar.com. Another example of a mashup that can be used to make money using the eBay API is http://www.ctxbay.com which provides contextual eBay ads for web publishers.

The aforementioned sites are just a few examples of the kinds of mashups that can be built with the eBay API. Since the API enables buying and listing of items for sale as well as obtaining inventory data from the service, one can build a very diverse set of applications.

(Via Dare Obasanjo aka Carnage4Life.)

Jonathan Boutelle has written about, what he calls, the Read/Write Div pattern.

What is the Read/Write Div pattern?

A new AJAX convention cropping up in a few places, one that is easy to implement and has real benefit to end users. I haven’t found a description of it anywhere, so I thought I’d write it up here.

The basic idea is that user controls (typically for editing the displayed data) should be hidden from the user until needed. At ‘rest’, an area of the screen displays information in read-only fashion.

Why the Read/Write Div Works

Fewer controls means that the user has to make fewer choices before taking an action, and therefore it takes less time for the user to choose which item to click on. The technical term for this effect is ‘Hicks Law’.

Also, the ‘read-only’ view of the data takes up much less space than the ‘read-write’ view does, so much more information can be packed into a given page, which means that the user doesn’t have to scroll down to read content. Avoiding the scrolling saves the user a measureable amount of time while browsing (3.05 seconds, according to my back-of-the-envelope GOMS keystroke analysis).

(Via Ajaxian Blog.)

(Via del.icio.us.)

Vinod Khosla: Web 2.0 2005: "Web 2.0 enriches online user experience by facilitating collaboration, participation, and communication. This is exciting investors once more and new Web 2.0 startups are finding it easy to get funding from venture capitalists. Although Vinod Khosla is a venture capitalist himself, he warns startups to learn the lessons of the failures of Web 1.0 companies and to use the money they raise judiciously and to remain creative rather than become comfortable with a business plan. [Web 2.0 audio from IT Conversations]"

(Via IT Conversations.)

(Via ProgrammableWeb.com.)

Ajax-S: Ajaxian slideshow software: "The idea came to me because I wanted a lightweight slideshow based on HTML, CSS and JavaScript, but I also wanted to separate the data of each page from the actual code that presents it. Therefore, I decided to move the data into an XML file and then use AJAX to retrieve it. The name AJAX-S is short for AJAX-Slides (or Asynchronous JavaScript and XML Slides, if you want to)."

(Via Ajaxian Blog.)

AJAX is clearly illuminating one of my pet issues: Separation of Application/Service Logic and Data. Even better, the concept of XML instance data is gradually getting much clearer. AJAX has created context for validating the concept of browser hosted Rich Internet Applications (RIA).

AJAX has become a widely accepted framework for the InternetOS that facilitates Rich Internet Application development using Web 2.0 (and beyond) APIs.

This Week’s Semantic Web:

"Ok, my first attempt at a round-up (in response to Phil’s observation of Planetary damage). Thanks to the conference there’s loads more here than there’s likely to be subsequent weeks, although it’s still only a fairly random sample and some of the links here are to heaps of other resources…
Incidentally, if anyone’s got a list/links for SemWeb-related blogs that aren’t on Planet RDF, I’d be grateful for a pointer. PS. Ok, I forget… are there any blogs that aren’t on Dave’s list yet..?

Quote of the week:

In the Semantic Web, it is not the Semantic which is new, it is the Web which is new.

- Chris Welty, IBM (lifted from TimBL’s slides)

Events

• 4th International Semantic Web Conference - happened this week, see : ISWC2005 Semantic Bank
• Semantic Desktop Workshop, 9-13 December 2005, Berlin
• Semantic Web Applications and Perspectives/Workshop (SWAP2005), 14-16 December, 2005
• Jena User Conference - May 10-11 2006, Bristol UK

Docs etc

• Conference highlights on the #swig chump: 2005-11-06, -07, -08, -09, -10; Ian’s notes; John’s resources; Leo’s stories; Uldis’ call to action; del.icio.us/iswc2005; flickr/iswc2005; foaf-moblog.
• Slides from Sir TimBL’s conference keynotes: Semantic Web for the Industry, Putting the Web back in Semantic Web
• Daniel Weitzner’s keynote: Privacy, Provenance, Property and Personhood
• Long-time SW researcher Stefan Decker now has a blog, inspirationally entitled Stefan Decker on the Semantic Web. (Stefan’s one of the head honchos at DERI). Sample snippet:
  

  I just noticed the article from Dan Zambonini ‘Is Web 2.0 killing the Semantic Web?‘. From my perspective the article shows a misconception that people seems to have around the Semantic Web: the Semantic Web effort itself is not provide applications (like the Web 2.0 meme indicates) - it rather provides standards to interlink applications.

• Leigh Dodds has two pieces demonstrating neat facilities offered by ARQ the SPARQL query API for Java: parameterised queries and extension functions.
• A new W3C Working Group has been chartered: Rule Interchange Format WG - ’ to produce a core rule language plus extensions which together allow rules to be translated between rule languages and thus transferred between rule systems.’. As noted by dajobe, phase 1 includes making a new XML syntax for RDF…
• UMBC Semantic Web Reference Card - if you only print one thing this year…or did you already do the SPARQL Reference card..?
• WebDescription - root wiki page for collecting notes on web description languages (ESW Wiki, announcement)
• Bot - IRC/Jabber chat bots that are either in use by Semantic Web developers or use Semantic Web technologies (ESW Wiki)
• microformat FAQs for RDF fans (ESW Wiki)
• W3C working draft : WSDL 2.0 - RDF Mapping
• SKOS (Simple Knowledge Organisation System) updated drafts: SKOS Core Vocabulary Specification, SKOS Core Guide
• working draft: SPARQL Protocol for RDF Using WSDL 1.1
• A relational algebra for SPARQL, Note on database layouts for SPARQL datastores (PDFs, Richard Cyganiak, HP)
• Amateur Fiction Online - The Web of Community Trust A Case Study in Community Focused Design for the SemanticWeb (PDF)
• Building a Semantic Wiki - IEEE article. See also: SemperWiki - Semantic Personal Wiki, WikSAR - Towards a Semantic Wiki Experience
  

Software and stuff

• Semantic Web Challenge applications (winner: CONFOTO - congrats bengee!)
• Piggy Bank 2.1.1 released.
• IRIS is a semantic desktop application framework that enables users to create a ‘personal map’ across their office-related information objects. IRIS includes a machine-learning platform to help automate this process. It provides ‘dashboard’ views, contextual navigation, and relationship-based structure across an extensible suite of office applications, including a calendar, web and file browser, e-mail client, and instant messaging client.
  (open source release due Jan 2006)
• MKSearch - ‘A new kind of search engine’ - RDF-backed (Sesame) with Web crawler, extracts and indexes metadata.
• FOAFRealm - Our goal is to design and implement D-FOAF, a distributed authentication and trust infrastructure without a centralised authority. D-FOAF will be a backbone for trust applications based on social relationships and will establish identity of users similar to the way we establish identify and trust in real life.
• Perl Net::Flickr::RDF
• WordPress SIOC (Semantically-Interlinked Online Communities) plugin updated (just copy wp-sioc.php into the root of your WP install and it just works)
• OntoMedia is intended for the representation of heterogenous media through description of the semantic content of that media. The representation may be limited to the description of some or all of the elements contained within the source or may include information regarding the narrative relationship that these elements have both to the media and to each other.
• mSpace is an interaction model to help explore relationships in information - ‘Imagine Google on iTunes’

Blog post title of the week:

Don’t give me that monkey-ass Web 1.0, either

- Uche Ogbuji

Also…a new threat to Semantic Web developers has been discovered: typhoid!, and the key to the Web’s full potential is…Tetris."

Solutions to allow XMLHttpRequest to talk to external services: "

Over on XML.com they published Fixing AJAX: XmlHttpRequest Considered Harmful.

This article discusses a few ways to get around the security constraints that we have to live with in the browsers theses days, in particular, only being able to talk to your domain via XHR.

The article walks you through three potential solutions:

1. Application proxies. Write an application in your favorite programming language that sits on your server, responds to XMLHttpRequests from users, makes the web service call, and sends the data back to users.
2. Apache proxy. Adjust your Apache web server configuration so that XMLHttpRequests can be invisibly re-routed from your server to the target web service domain.
3. Script tag hack with application proxy (doesn't use XMLHttpRequest at all). Use the HTML script tag to make a request to an application proxy (see #1 above) that returns your data wrapped in JavaScript. This approach is also known as On-Demand JavaScript.

I can't wait for Trusted Relationships within the browser - server infrastructure.

With respect to Apache proxies, these things are priceless. I recently talked about them in relation to Migrating data centers with zero downtime.

What do you guys think about this general issue? Have you come up with any interesting solutions? Any ideas on how we can keep security, yet give us the freedom that we want?

(Via Ajaxian Blog.)

Well here is what I think (actually know):

Our Virtuoso Universal Server has been sitting waiting to deliver this for years (for the record see the Virtuoso 2000 Press Release). Virtuoso can proxy for disparate data sources and expose disparate data as Well-Formed XML using an array of vocabularies (you experience this SQL-XML integration on the fly every time you interact with various elements of my public blog).

Virtuoso has always been able to expose Application Logic as SOAP and/or RESTful/RESTian style XML Web Services. This blog's search page is a simple demo of this capability.

Virtuoso is basically a Junction Box / Aggregator / Proxy for disparate Data, Applications, Services, and BPEL compliant business processes. AJAX clients talk to this single multi-purpose server which basically acts as a conduit to content/data, services, and processes (which are composite services).

BTW - there is a lot more, but for now, thou shall have to seek in order to find :-)

Answer: NO.

Comparing Web 2.0 to Windows is like comparing Apples and Oranges!

The Internet displaced Windows (long time ago!). The effect of this reality is simply working its way through Geoffrey Moore's Bell Curve - in "left to right" fashion. By the way, there isn't a single thing Microsoft can do about this beyond accepting this reality and gearing itself up to compete as best it can in this new reality.

The Internet is the Operating System for the New Computer - aptly coined: "The Network" by Sun years ago (unfortunately a blind preoccupation with Java has completely obscured Sun's fundamental vision regarding this matter).

Web 2.0 provides the Windows API equivalent for the InternetOS.

As you can see Web 2.0 and the Semantic Web are mutually inclusive paradigms as reemphasized via this additional "mashup" (I don't really like the word "mashup", especially as it isn't different from/than "repurposing"?)

Quick Example using my blog:

1. My Blog's Search Page (note it support Full Text and XPath/XQuery)

2. Search on pattern 'Web 2.0' via my Blog's Search Engine

3. Hactivism" regarding this matter. Certainly worth a full-post-scrape for my ongoing content annotation efforts (see Linkblog and BlogSummary).

Digest the rest of Dare's post:

Clone the Google APIs: Kill That Noise: "

Yesterday Dave Winer wrote in a post about cloning the Google API Dave Winer wrote

Let's make the Google API an open standard. Back in 2002, Google took a bold first step to enable open architecture search engines, by creating an API that allowed developers to build applications on top of their search engine. However, there were severe limits on the capacity of these applications. So we got a good demo of what might be, now three years later, it's time for the real thing.



and earlier that


If you didn't get a chance to hear yesterday's podcast, it recommends that Microsoft clone the Google API for search, without the keys, and without the limits. When a developer's application generates a lot of traffic, buy him a plane ticket and dinner, and ask how you both can make some money off their excellent booming application of search. This is something Google can't do, because search is their cash cow. That's why Microsoft should do it. And so should Yahoo. Also, there's no doubt Google will be competing with Apple soon, so they should be also thinking about ways to devalue Google's advantage.

This doesn't seem like a great idea to me for a wide variety of reasons but first, let's start with a history lesson before I tackle this specific issue

A Trip Down Memory Lane
This history lesson used to be in is in a post entitled The Tragedy of the API by Evan Williams but seems to be gone now. Anyway, back in the early days of blogging the folks at Pyra [which eventually got bought by Google] created the Blogger API for their service. Since Blogspot/Blogger was a popular service, a the number of applications that used the API quickly grew. At this point Dave Winer decided that since the Blogger API was so popular he should implement it in his weblogging tools but then he decided that he didn't like some aspects of it such as application keys (sound familiar?) and did without them in his version of the API. Dave Winer's version of the Blogger API became the MetaWeblog API. These APIs became de facto standards and a number of other weblogging applications implemented them.

After a while, the folks at Pyra decided that their API needed to evolve due to various flaws in its design. As Diego Doval put it in his post a review of blogging APIs, The Blogger API is a joke, and a bad one at that. This lead to the creation of the Blogger API 2.0. At this point a heated debate erupted online where Dave Winer berated the Blogger folks for deviating from an industry standard. The irony of flaming a company for coming up with a v2 of their own API seemed to be lost on many of the people who participated in the debate. Eventually the Blogger API 2.0 went nowhere.

Today the blogging API world is a few de facto standards based on a hacky API created by a startup a few years ago, a number of site specific APIs (LiveJournal API, MovableType API, etc) and a number of inconsistently implemented versions of the Atom API.


On Cloning the Google Search API
To me the most salient point in the hijacking of the Blogger API from Pyra is that it didn't change the popularity of their service or even make Radio Userland (Dave Winer's product) catch up to them in popularity. This is important to note since this is Dave Winer's key argument for Microsoft cloning the Google API.

Off the top of my head, here are my top three technical reasons for Microsoft to ignore the calls to clone the Google Search APIs


1. Difference in Feature Set: The features exposed by the API do not run the entire gamut of features that other search engines may want to expose. Thus even if you implement something that looks a lot like the Google API, you'd have to extend it to add the functionality that it doesn't provide. For example, compare the features provided by the Google API to the features provided by the Yahoo! search API. I can count about half a dozen features in the Yahoo! API that aren't in the Google API.

2. Difference in Technology Choice: The Google API uses SOAP. This to me is a phenomenally bad technical decision because it raises the bar to performing a basic operation (data retrieval) by using a complex technology. I much prefer Yahoo!'s approach of providing a RESTful API and MSN Windows Live Search's approach of providing RSS search feeds and a SOAP API for the folks who need such overkill.
   

3. Unreasonable Demands: A number of Dave Winer's demands seem contradictory. He asks companies to not require application keys but then advises them to contact application developers who've built high traffic applications about revenue sharing. Exactly how are these applications to be identified without some sort of application ID? As for removing the limits on the services? I guess Dave is ignoring the fact that providing services costs money, which I seem to remember is why he sold weblogs.com to Verisign for a few million dollars. I do agree that some of the limits on existing search APIs aren't terribly useful. The Google API limit of 1000 queries a day seems to guarantee that you won't be able to power a popular application with the service.
   

4. Lack of Innovation: Copying Google sucks.
   

(Via Dare Obasanjo aka Carnage4Life.)

Stop whatever you are doing ...: "

(Spotted Via The Obvious?.)

Yes, the ramifications are deep! Tom Coates' screencast demonstrates an internal variation of an activity that is taking place on many fronts (concurrently) across the NET. I tend to refer to this effort as "Self Annotation"; the very process that will ultimately take us straight to "Semantic Web". It is going to happen much quicker than anticipated because technology is taking the pain out of metadata annotation (e.g. what you do when you tag everything that is ultimately URI accessible). Technology is basically delivering what Jon Udell calls: "reducing the activation threshold".

Using my comments above for context placement, I suggest you take a look at, or re-read Jon Udell's post titled: Many Meanings of Metadata.

Once again, the Web 2.0 brouhaha (in every sense of the word) is a reaction to a critical inflection that ultimately transitions the "Semantic Web" from "Mirage" to "Nirvana". Put differently (with humor in mind solely!), Web 2.0 is what I tend to call a "John the Baptist" paradigm, and we all know what happened to him :-)

Web 2.0 is a conduit to a far more important destination. The tendency to treat Web 2.0 as a destination rather than a conduit has contributed to the recent spate of Bozo bit flipping posts all over the blogosphere (is this an attempt to behead John, metaphorically speaking?). Humor aside, a really important thing about the Web 2.0 situation is that when we make the quantum evolutionary leap (internet time, mind you) to the "Semantic Web" (or whatever groovy name we dig up for it in due course) we will certainly have a plethora of reference points (I mean Web 2.0 URIs) ensuring that we do not revisit the "Missing Link" evolutionary paradox :-)

BTW - You can see some example of my contribution to the ongoing annotation process by looking at:

My Blog Summary Page

My Linkblog

My Blog Search

My Blog Query Service (click on the enhanced view if you're a SOAP geek; also note blogid=127)

"...Also today I came across the latest project of a man who wants to tear down Tim Berners-Lee's World Wide Web and replace it with his own vision. It used to be known as Xanadu, but has since morphed into Transliterature, A Humanist Design. I am of course referring to Ted Nelson, who invented the term 'hypertext' in 1965 and is generally regarded as a computing pioneer.

Ted Nelson recently wrote an essay about 'Indirect Documents', which got Slashdotted today. In the essay Nelson outlines why (in his opinion) the Xanadu project failed and he explains his new vision for Transliterature. He takes a number of potshots at Tim Berners-Lee's WWW on the way, e.g.:

'Why don't I like the web? I hate its flapping and screeching and emphasis on appearance; its paper-simulation rectangles of Valuable Real Estate, artifically created by the NCSA browser, now hired out to advertisers; its hierarchies exposed and imposed; its untyped one-way links only from inside the document. (The one-way links hidden under text were a regrettable simplification of hypertext which I assented to in '68 on the HES project. But that's another story.) Only trivial links are possible; there is nothing to support careful annotation and study; and, of course, there is no transclusion.'

Ted Nelson is certainly an original and I'm glad he's still around to throw spanners in the works. I've written about him before and I'm sure I will again, Web 2.0 or not.

(Excerpted From: Read/Write Web.)

My thoughts on the commentary above:

There is nothing fundamentally incompatible between Ted Nelson's pursuits and future incarnation's of the Web. None whatsoever -- we are simply working our way through an process. The process in question is what I call "standards driven ubiquity" (becoming de facto at Internet Speed). Remember Sun's "The Network is the Computer" vision? Well, without a "Computer" in mind-space you can't think in terms of "Operating Systems". Thats all changing, because today we are gradually beginning to accept the imminent reality that "The Internet is the Operating System" and not Windows/UNIX/Mac OS X/Others. Ahem! And after the Operating System what comes next? I think a set of Application Programming Interfaces (APIs), and I think we know what that is (in all of its controversial glory), the very thing we refer to as Web 2.0 (the APIs for the Internet Operating System).

Note: In addition to the Computer, Operating System, and Application Programming Interfaces, we also have those frequently misunderstood and under-appreciated workhorses called "Databases" in place (but we still call them Web Sites for now). And by the way, "Internet Filesystem" has been there forever, but for some reason we can't see WebDAV in all its current and future glory (that will change very soon also!).

Ted and TBL are cool with each (whether they know it or not)! I see no mutual exclusivity in their collective visions (IMHO) :-)

Anyway, Marc's article is a very refreshing read because it provides a really good insight into the general landscape of a rapidly evolving Web alongside genuine appreciation of our broader timeless pursuit of "Openness".

To really help this document provide additional value have scrapped the content of the original post and dumped it below so that we can appreciate the value of the links embedded within the article (note: thanks to Virtuoso I only had to paste the content into my blog, the extraction to my Linkblog and Blog Summary Pages are simply features of my Virtuoso based Blog Engine):

Breaking the Web Wide Open! (complete story)

Even the web giants like AOL, Google, MSN, and Yahoo need to observe these open standards, or they'll risk becoming the "walled gardens" of the new web and be coolio no more.

Marc Canter [Broadband Mechanics, Inc.] | POSTED: 09.26.05 @12:00

Editorial Note: Several months ago, AlwaysOn got a personal invitation from Yahoo founder Jerry Yang "to see and give us feedback on our new social media product, y!360." We were happy to oblige and dutifully showed up, joining a conference room full of hard-core bloggers and new, new media types. The geeks gave Yahoo 360 an overwhelming thumbs down, with comments like, "So the only services I can use within this new network are Yahoo services? What if I don't use Yahoo IM?" In essence, the Yahoo team was booed for being "closed web," and we heartily agreed. With Yahoo 360, Yahoo continues building its own "walled garden" to control its 135 million customers—an accusation also hurled at AOL in the early 1990s, before AOL migrated its private network service onto the web. As the  Economist recently noted, "Yahoo, in short, has old media plans for the new-media era." The irony to our view here is, of course, that today's AO Network is also a "closed web." In the end, Mr. Yang's thoughtful invitation and our ensuing disappointment in his new service led to the assignment of this article. It also confirmed our existing plan to completely revamp the AO Network around open standards. To tie it all together, we recruited the chief architect of our new site, the notorious Marc Canter, to pen this piece. We look forward to our reader feedback. Breaking the Web Wide Open! By Marc Canter For decades, "walled gardens" of proprietary standards and content have been the strategy of dominant players in mainframe computer software, wireless telecommunications services, and the World Wide Web—it was their successful lock-in strategy of keeping their customers theirs. But like it or not, those walls are tumbling down. Open web standards are being adopted so widely, with such value and impact, that the web giants—Amazon, AOL, eBay, Google, Microsoft, and Yahoo—are facing the difficult decision of opening up to what they don't control. The online world is evolving into a new open web (sometimes called the Web 2.0), which is all about being personalized and customized for each user. Not only open source software, but open standards are becoming an essential component. Many of the web giants have been using open source software for years. Most of them use at least parts of the LAMP (Linux, Apache, MySQL, Perl/Python/PHP) stack, even if they aren't well-known for giving back to the open source community. For these incumbents that grew big on proprietary web services, the methods, practices, and applications of open source software development are difficult to fully adopt. And the next open source movements—which will be as much about open standards as about code—will be a lot harder for the incumbents to exploit. While the incumbents use cheap open source software to run their back-ends systems, their business models largely depend on proprietary software and algorithms. But our view a new slew of open software, open protocols, and open standards will confront the incumbents with the classic Innovator's Dilemma.  Should they adopt these tools and standards, painfully cannibalizing their existing revenue for a new unproven concept, or should they stick with their currently lucrative model with the risk that eventually a bunch of upstarts eat their lunch? Credit should go to several of the web giants who have been making efforts to "open up." Google, Yahoo, eBay, and Amazon all have Open APIs (Application Programming Interfaces) built into their data and systems. Any software developer can access and use them for whatever creative purposes they wish. This means that the API provider becomes an open platform for everyone to use and build on top of. This notion has expanded like wildfire throughout the blogosphere, so nowadays, Open APIs are pretty much required. Other incumbents also have open strategies. AOL has got the RSS religion, providing a feedreader and RSS search in order to escape the "walled garden of content" stigma. Apple now incorporates podcasts, the "personal radio shows" that are latest rage in audio narrowcasting, into iTunes. Even Microsoft is supporting open standards, for example by endorsing SIP (Session Initiation Protocol) for internet telephony and conferencing over Skype's proprietary format or one of its own devising. But new open standards and protocols are in use, under construction, or being proposed every day, pushing the envelope of where we are right now. Many of these standards are coming from startup companies and small groups of developers, not from the giants. Together with the Open APIs, those new standards will contribute to a new, open infrastructure. Tens of thousands of developers will use and improve this open infrastructure to create new kinds of web-based applications and services, to offer web users a highly personalized online experience. A Brief History of Openness At this point, I have to admit that I am not just a passive observer, full-time journalist or "just some blogger"—but an active evangelist and developer of these standards. It's the vision of "open infrastructure" that's driving my company and the reason why I'm writing this article. This article will give you some of the background behind on these standards, and what the evolution of the next generation of open standards will look like. Starting back in the 1980s, establishing a software standard was a key strategy for any software company. My former company, MacroMind (which became Macromedia), achieved this goal early on with Director. As Director evolved into Flash, the world saw that other companies besides Microsoft, Adobe, and Apple could establish true cross-platform, independent media standards. Then Tim Berners-Lee and Marc Andreessen came along, and changed the rules of the software business and of entrepreneurialism. No matter how entrenched and "standardized" software was, the rug could still get pulled out from under it. Netscape did it to Microsoft, and then Microsoft did it back  to Netscape. The web evolved, and lots of standards evolved with it. The leading open source standards (such as the LAMP stack) became widely used alternatives to proprietary closed-source offerings. Open standards are more than just technology. Open standards mean sharing, empowering, and community support. Someone floats a new idea (or meme) and the community runs with it – with each person making their own contributions to the standard – evolving it without a moment's hesitation about "giving away their intellectual property." One good example of this was Dave Sifry, who built the Technorati blog-tracking technology inspired by the Blogging Ecosystem, a weekend project by young hacker Phil Pearson. Dave liked what he saw and he ran with it—turning Technorati into what it is today. Dave Winer has contributed enormously to this area of open standards. He defined and personally created several open standards and protocols—such as RSS, OPML, and XML-RPC. Dave has also helped build the blogosphere through his enthusiasm and passion. By 2003, hundreds of programmers were working on creating and establishing new standards for almost everything. The best of these new standards have evolved into compelling web services platforms – such as del.icio.us, Webjay, or Flickr. Some have even spun off formal standards – like XSPF (a standard for playlists) or instant messaging standard XMPP (also known as Jabber). Today's Open APIs are complemented by standardized Schemas—the structure of the data itself and its associated meta-data. Take for example a podcasting feed. It consists of: a) the radio show itself, b) information on who is on the show, what the show is about and how long the show is (the meta-data) and also c) API calls to retrieve a show (a single feed item) and play it from a specified server. The combination of Open APIs, standardized schemas for handling meta-data, and an industry which agrees on these standards are breaking the web wide open right now. So what new open standards should the web incumbents—and you—be watching? Keep an eye on the following developments: Identity Attention Open Media Microcontent Publishing Open Social Networks Tags Pinging Routing Open Communications Device Management and Control 1. Identity Right now, you don't really control your own online identity. At the core of just about every online piece of software is a membership system. Some systems allow you to browse a site anonymously—but unless you register with the site you can't do things like search for an article, post a comment, buy something, or review it. The problem is that each and every site has its own membership system. So you constantly have to register with new systems, which cannot share data—even you'd want them to. By establishing a "single sign-on" standard, disparate sites can allow users to freely move from site to site, and let them control the movement of their personal profile data, as well as any other data they've created. With Passport, Microsoft unsuccessfully attempted to force its proprietary standard on the industry. Instead, a world is evolving where most people assume that users want to control their own data, whether that data is their profile, their blog posts and photos, or some collection of their past interactions, purchases, and recommendations. As long as users can control their digital identity, any kind of service or interaction can be layered on top of it. Identity 2.0 is all about users controlling their own profile data and becoming their own agents. This way the users themselves, rather than other intermediaries, will profit from their ID info. Once developers start offering single sign-on to their users, and users have trusted places to store their data—which respect the limits and provide access controls over that data, users will be able to access personalized services which will understand and use their personal data. Identity 2.0 may seem like some geeky, visionary future standard that isn't defined yet, but by putting each user's digital identity at the core of all their online experiences, Identity 2.0 is becoming the cornerstone of the new open web. The Initiatives: Right now, Identity 2.0 is under construction through various efforts from Microsoft (the "InfoCard" component built into the Vista operating system and its "Identity Metasystem"), Sxip Identity, Identity Commons, Liberty Alliance, LID (NetMesh's Lightweight ID), and SixApart's OpenID. More Movers and Shakers: Identity Commons and Kaliya Hamlin, Sxip Identity and Dick Hardt, the Identity Gang and Doc Searls, Microsoft's Kim Cameron, Craig Burton, Phil Windley, and Brad Fitzpatrick, to name a few. 2. Attention How many readers know what their online attention is worth? If you don't, Google and Yahoo do—they make their living off our attention. They know what we're searching for, happily turn it into a keyword, and sell that keyword to advertisers. They make money off our attention. We don't. Technorati and friends proposed an attention standard, Attention.xml, designed to "help you keep track of what you've read, what you're spending time on, and what you should be paying attention to." AttentionTrust is an effort by Steve Gillmor and Seth Goldstein to standardize on how captured end-user performance, browsing, and interest data are used. Blogger Peter Caputa gives a good summary of AttentionTrust: "As we use the web, we reveal lots of information about ourselves by what we pay attention to. Imagine if all of that information could be stored in a nice neat little xml file. And when we travel around the web, we can optionally share it with websites or other people. We can make them pay for it, lease it ... we get to decide who has access to it, how long they have access to it, and what we want in return. And they have to tell us what they are going to do with our Attention data." So when you give your attention to sites that adhere to the AttentionTrust, your attention rights (you own your attention, you can move your attention, you can pay attention and be paid for it,  and you can see how your attention is used) are guaranteed. Attention data is crucial to the future of the open web, and Steve and Seth are making sure that no one entity or oligopoly controls it. Movers and Shakers: Steve Gillmor, Seth Goldstein, Dave Sifry and the other Attention.xml folks. 3. Open Media Proprietary media standards—Flash, Windows Media, and QuickTime, to name a few —helped liven up the web. But they are proprietary standards that try to keep us locked in, and they weren't created from scratch to handle today's online content. That's why, for many of us, an Open Media standard has been a holy grail. Yahoo's new Media RSS standard brings us one step closer to achieving open media, as do Ogg Vorbis audio codecs, XSPF playlists, or MusicBrainz. And several sites offer digital creators not only a place to store their content, but also to sell it. Media RSS (being developed by Yahoo with help from the community) extends RSS and combines it with "RSS enclosures" —adds metadata to any media item—to create a comprehensive solution for media "narrowcasters." To gain acceptance for Media RSS, Yahoo knows it has to work with the community. As an active member of this community, I can tell you that we'll create Media RSS equivalents for rdf (an alternative subscription format) and Atom (yet another  subscription format), so no one will be able to complain that Yahoo is picking sides in format wars. When Yahoo announced the purchase of Flickr, Yahoo founder Jerry Yang insinuated that Yahoo is acquiring "open DNA" to turn Yahoo into an open standards player. Yahoo is showing what happens when you take a multi-billion dollar company and make openness one of its core values—so Google, beware, even if Google does have more research fellows and Ph.D.s. The open media landscape is far and wide, reaching from game machine hacks and mobile phone downloads to PC-driven bookmarklets, players, and editors, and it includes many other standardization efforts. XSPF is an open standard for playlists, and MusicBrainz is an alternative to the proprietary (and originally effectively stolen) database that Gracenote licenses. Ourmedia.org is a community front-end to Brewster Kahle's Internet Archive. Brewster has promised free bandwidth and free storage forever to any content creators who choose to share their content via the Internet Archive. Ourmedia.org is providing an easy-to-use interface and community to get content in and out of the Internet Archive, giving ourmedia.org users the ability to share their media anywhere they wish, without being locked into a particular service or tool. Ourmedia plans to offer open APIs and an open media registry that interconnects other open media repositories into a DNS-like registry (just like the www domain system), so folks can browse and discover open content across many open media services. Systems like Brightcove and Odeo support the concept of an open registry, and hope to work with digital creators to sell their work to fulfill the financial aspect of the "Long Tail." More Movers and Shakers: Creative Commons, the Open Media Network, Jay Dedman, Ryanne Hodson, Michael Verdi, Eli Chapman, Kenyatta Cheese, Doug Kaye, Brad Horowitz, Lucas Gonze, Robert Kaye, Christopher Allen, Brewster Kahle, JD Lasica, and indeed, Marc Canter, among others. 4. Microcontent Publishing Unstructured content is cheap to create, but hard to search through. Structured content is expensive to create, but easy to search. Microformats resolve the dilemma with simple structures that are cheap to use and easy to search. The first kind of widely adopted microcontent is blogging. Every post is an encapsulated idea, addressable via a URL called a permalink. You can syndicate or subscribe to this microcontent using RSS or an RSS equivalent, and news or blog aggregators can then display these feeds in a convenient readable fashion. But a blog post is just a block of unstructured text—not a bad thing, but just a first step for microcontent. When it comes tostructured data, such as personal identity profiles, product reviews, or calendar-type event data, RSS was not designed to maintain the integrity of the structures. Right now, blogging doesn't have the underlying structure necessary for full-fledged microcontent publishing. But that will change. Think of local information services (such as movie listings, event guides, or restaurant reviews) that any college kid can access and use in her weekend programming project to create new services and tools. Today's blogging tools will evolve into microcontent publishing systems, and will help spread the notion of structured data across the blogosphere. New ways to store, represent and produce microcontent will create new standards, such as Structured Blogging and Microformats. Microformats differ from RSS feeds in that you can't subscribe to them. Instead, Microformats are embedded into webpages and discovered by search engines like Google or Technorati. Microformats are creating common definitions for "What is a review or event? What are the specific fields in the data structure?" They can also specify what we can do with all this information.OPML (Outline Processor Markup Language) is a hierarchical file format for storing microcontent and structured data. It was developed by Dave Winer of RSS and podcast fame. Events are one popular type of microcontent. OpenEvents is already working to create shared databases of standardized events, which would get used by a new generation of event portals—such as Eventful/EVDB, Upcoming.org, and WhizSpark. The idea of OpenEvents is that event-oriented systems and services can work together to establish shared events databases (and associated APIs) that any developer could then use to create and offer their own new service or application. OpenReviews is still in the conceptual stage, but it would make it possible to provide open alternatives to closed systems like Epinions, and establish a shared database of local and global reviews. Its shared open servers would be filled with all sorts of reviews for anyone to access. Why is this important? Because I predict that in the future, 10 times more people will be writing reviews than maintaining their own blog. The list of possible microcontent standards goes on: OpenJobpostings, OpenRecipes, and even OpenLists. Microsoft recently revealed that it has been working on an important new kind of microcontent: Lists—so OpenLists will attempt to establish standards for the kind of lists we all use, such as lists of Links, lists of To Do Items, lists of People, Wish Lists, etc. Movers and Shakers: Tantek Çelik and Kevin Marks of Technorati, Danny Ayers, Eric Meyer, Matt Mullenweg, Rohit Khare, Adam Rifkin, Arnaud Leene, Seb Paquet, Alf Eaton, Phil Pearson, Joe Reger, Bob Wyman among others. 5. Open Social Networks I'll never forget the first time I met Jonathan Abrams, the founder of Friendster. He was arrogant and brash and he claimed he "owned"  all his users, and that he was going to monetize them and make a fortune off them. This attitude robbed Friendster of its momentum, letting MySpace, Facebook, and other social networks take Friendster's place. Jonathan's notion of social networks as a way to control users is typical of the Web 1.0 business model and its attitude towards users in general. Social networks have become one of the battlegrounds between old and new ways of thinking. Open standards for Social Networking will define those sides very clearly. Since meeting Jonathan, I have been working towards finding and establishing open standards for social networks. Instead of closed, centralized social networks with 10 million people in them, the goal is making it possible to have 10 million social networks that each have 10 people in them. FOAF (which stands for Friend Of A Friend, and describes people and relationships in a way that computers can parse) is a schema to represent not only your personal profile's meta-data, but your social network as well. Thousands of researchers use the FOAF schema in their "Semantic Web" projects to connect people in all sorts of new ways. XFN is a microformat standard for representing your social network, while vCard (long familiar to users of contact manager programs like Outlook) is a microformat that contains your profile information. Microformats are baked into any xHTML webpage, which means thatany blog, social network page, or any webpage in general can "contain" your social network in it—and be used byany compatible tool, service or application. PeopleAggregator is an earlier project now being integrated into open content management framework Drupal. The PeopleAggregator APIs will make it possible to establish relationships, send messages, create or join groups, and post between different social networks. (Sneak preview: this technology will be available in the upcoming GoingOn Network.) All of these open social networking standards mean that inter-connected social networks will form a mesh that will parallel the blogosphere. This vibrant, distributed, decentralized world will be driven by open standards: personalized online experiences are what the new open web will be all about—and what could be more personalized than people's networks? Movers and Shakers: Eric Sigler, Joel De Gan, Chris Schmidt, Julian Bond, Paul Martino, Mary Hodder, Drummond Reed, Dan Brickley, Randy Farmer, and Kaliya Hamlin, to name a few. 6. Tags Nowadays, no self-respecting tool or service can ship without tags. Tags are keywords or phrases attached to photos, blog posts, URLs, or even video clips. These user- and creator-generated tags are an open alternative to what used to be the domain of librarians and information scientists: categorizing information and content using taxonomies. Tags are instead creating "folksonomies." The recently proposed OpenTags concept would be an open, community-owned version of the popular Technorati Tags service. It would aggregate the usage of tags across a wide range of services, sites, and content tools. In addition to Technorati's current tag features, OpenTags would let groups of people share their tags in "TagClouds." Open tagging is likely to include some of the open identity features discussed above, to create a tag system that is resilient to spam, and yet trustable across sites all over the web. OpenTags owes a debt to earlier versions of shared tagging systems, which include Topic Exchange and something called the k-collector—a knowledge management tag aggregator—from Italian company eVectors. Movers & Shakers: Phil Pearson, Matt Mower , Paolo Valdemarin, and Mary Hodder and Drummond Reed again, among others. 7. Pinging Websites used to be mostly static. Search engines that crawled (or "spidered") them every so often did a good enough job to show reasonably current versions of your cousin's homepage or even Time magazine's weekly headlines. But when blogging took off, it became hard for search engines to keep up. (Google has only just managed to offer blog-search functionality, despite buying Blogger back in early 2003.) To know what was new in the blogosphere, users couldn't depend on services that spidered webpages once in a while. The solution: a way for blogs themselves to automatically notify blog-tracking sites that they'd been updated. Weblogs.com was the first blog "ping service": it displayed the name of a blog whenever that blog was updated. Pinging sites helped the blogosphere grow, and more tools, services, and portals started using pinging in new and different ways. Dozens of pinging services and sites—most of which can't talk to each other—sprang up. Matt Mullenweg (the creator of open source blogging software WordPress) decided that a one-stop service for pinging was needed. He created Ping-o-Matic—which aggregates ping services and simplifies the pinging process for bloggers and tool developers. With Ping-o-Matic, any developer can alert all of the industry's blogging tools and tracking sites at once. This new kind of open standard, with shared infrastructure, is a critical to the scalability of Web 2.0 services. As Matt said: There are a number of services designed specifically for tracking and connecting blogs. However it would be expensive for all the services to crawl all the blogs in the world all the time. By sending a small ping to each service you let them know you've updated so they can come check you out. They get the freshest data possible, you don't get a thousand robots spidering your site all the time. Everybody wins. Movers and Shakers: Matt Mullenweg, Jim Winstead, Dave Winer 8. Routing Bloggers used to have to manually enter the links and content snippets of blog posts or news items they wanted to blog. Today, some RSS aggregators can send a specified post directly into an associated blogging tool: as bloggers browse through the feeds they subscribe to, they can easily specify and send any post they wish to "reblog" from their news aggregator or feed reader into their blogging tool. (This is usually referred to as "BlogThis.") As structured blogging comes into its own (see the section on Microcontent Publishing), it will be increasingly important to maintain the structural integrity of these pieces of microcontent when reblogging them. Promising standard RedirectThis will combine a "BlogThis"-like capability while maintaining the integrity of the microcontent. RedirectThis will let bloggers and content developers attach a simple "PostThis" button to their posts. Clicking on that button will send that post to the reader/blogger's favorite blogging tool. This favorite tool is specified at the RedirectThis web service, where users register their blogging tool of choice. RedirectThis also helps maintain the integrity and structure of microcontent—then it's just up to the user to prefer a blogging tool that also attains that lofty goal of microcontent integrity. OutputThis is another nascent web services standard, to let bloggers specify what "destinations" they'd like to have as options in their blogging tool. As new destinations are added to the service, more checkboxes would get added to their blogging tool—allowing them to route their published microcontent to additional destinations. Movers and Shakers: Michael Migurski, Lucas Gonze 9. Open Communications Likely, you've experienced the joys of finding friends on AIM or Yahoo Messenger, or the convenience of Skyping with someone overseas. Not that you're about to throw away your mobile phone or BlackBerry, but for many, also having access to Instant Messaging (IM) and Voice over IP (VoIP) is crucial. IM and VoIP are mainstream technologies that already enjoy the benefits of open standards. Entire industries are born—right this second—based around these open standards. Jabber has been an open IM technology for years—in fact, as XMPP, it was officially dubbed a standard by the IETF. Although becoming an official IETF standard is usually the kiss of death, Jabber looks like it'll be around for a while, as entire generations of collaborative, work-group applications and services have been built on top of its messaging protocol. For VoIP, Skype is clearly the leading standard today—though one could argue just how "open" it is (and defenders of the IETF's SIP standard often do). But it is free and user-friendly, so there won't be much argument from users  about it being insufficiently open. Yet there may be a cloud on Skype's horizon: web behemoth Google recently released a beta of Google Talk, an IM client committed to open standards. It currently supports XMPP, and will support SIP for VoIP calls. Movers and Shakers: Jeremie Miller, Henning Schulzrinne, Jon Peterson, Jeff Pulver 10. Device Management and Control To access online content, we're using more and more devices. BlackBerrys, iPods, Treos, you name it. As the web evolves, more and more different devices will have to communicate with each other to give us the content we want when and where we want it. No-one wants to be dependent on one vendor anymore—like, say, Sony—for their laptop, phone, MP3 player, PDA, and digital camera, so that it all works together. We need fully interoperable devices, and the standards to make that work. And to fully make use of how content is moving online content and innovative web services, those standards need to be open. MIDI (musical instrument digital interface), one of the very first open standards in music, connected disparate vendors' instruments, post-production equipment, and recording devices. But MIDI is limited, and MIDI II has been very slow to arrive. Now a new standard for controlling musical devices has emerged: OSC (Open SoundControl). This protocol is optimized for modern networking technology and inter-connects music, video and controller devices with "other multimedia devices." OSC is used by a wide range of developers, and is being taken up in the mainstream MIDI marketplace. Another open-standards-based device management technology is ZigBee, for building wireless intelligence and network monitoring into all kinds of devices. ZigBee is supported by many networking, consumer electronics, and mobile device companies.       · · · · · ·     The Change to Openness The rise of open source software and its "architecture of participation" are completely shaking up the old proprietary-web-services-and-standards approach. Sun Microsystems—whose proprietary Java standard helped define the Web 1.0—is opening its Solaris OS and has even announced the apparent paradox of an open-source Digital Rights Management system. Today's incumbents will have to adapt to the new openness of the Web 2.0. If they stick to their proprietary standards, code, and content, they'll become the new walled gardens—places users visit briefly to retrieve data and content from enclosed data silos, but not where users "live." The incumbents' revenue models will have to change. Instead of "owning" their users, users will know they own themselves, and will expect a return on their valuable identity and attention. Instead of being locked into incompatible media formats, users will expect easy access to digital content across many platforms. Yesterday's web giants and tomorrow's users will need to find a mutually beneficial new balance—between open and proprietary, developer and user, hierarchical and horizontal, owned and shared, and compatible and closed. Marc Canter is an active evangelist and developer of open standards. Early in his career, Marc founded MacroMind, which became Macromedia. These days, he is CEO of Broadband Mechanics, a founding member of the Identity Gang and of ourmedia.org. Broadband Mechanics is currently developing the GoingOn Network (with the AlwaysOn Network), as well as an open platform for social networking called the PeopleAggregator. A version of the above post appears in the Fall 2005 issue of AlwaysOn's quarterly print blogozine, and ran as a four-part series on the AlwaysOn Network website.

(Via Marc's Voice.)

David Cowan is a partner at Bessemer Venture Partners and writes a blog called Who Has Time For This. He’s on this list partially because he incubated the hottest and most anticipated company on the web right now, Flock.

Tim Draper invested in Skype. Done. He also sits on the board of SocialText, and his fund was in Baidu.

David Hornik is is a General Partner at August Capital and writes a blog that has over 10,000 RSS readers.
Josh Kopelman, through FirstRoundCapital, is quietly filtering through just about every young web 2.0 company, and investing in many of them.

Fred Wilson is a founding partner of Union Square Ventures and writes the extremely popular A VC. If you are new to web 2.0, start with his Blogging 1.0 post.

Jeff Clavier - Jeff is a former VC and still makes the odd angel investment (Feedster, Truveo, and a few others). His new venture allows him to work with pre-funding companies and get them ready for prime time.

Brad Feld - Brad is a managing director at Mobius Venture Capital and writes a must-read web 2.0 blog called Feld Thoughts. Read his posts on Term Sheets if you are in the process of raising capital.

O’Reilly AlphaTech Ventures - This is the only non-person on here. OATV just closed a $50 million fund to invest in young companies. Given the incredible access Tim O’Reilly has to these companies, OATV could quickly become an important fund in the web 2.0 space.

Pierre Omidyar - Pierre founded ebay and is the Co-founder of Omidyar Network, where he’s invested in a number of interesting companies including EVDB, SocialText and Feedster, and others.

Peter Rip - Peter is a founding partner of Leapfrog Ventures, a $100 million fund. Peter also writes Early Stage VC, another must-read blog. His investments include ojos, an incredible new photo-metadata service that is going to be extremely disruptive (and useful).

Peter Thiel - Peter, the former CEO of paypal, has invested in LinkedIn, Friendster, LinkedIn and other web 2.0 companies. He’s just created the Founders Fund.

Thomas Ball - Tom is a Venture Partner at Austin Ventures, a fund with $3 billion under management. He’s their consumer and web 2.0 guy and seems to be spending a lot of time in Silicon Valley and at web 2.0 event.

Dan Grossman - Dan is a principal at Venrock Associates and has recently started a great blog called A Venture Forth (where he wrote a much bookmarked post on Ajax).

Jason Pressman - Jason is a principal at Shasta Ventures, a young $200 million fund that has a deep commitment to and expertise in consumer-focused businesses.

Web 2.0: Conversation with Vinod Khosla:

Vinod started by explaining that, contrary to the rumor, he is not starting a fund, he is just investing his own capital - hiring a few people to help with this. He was a General Partner in all Kleiner funds he was involved in, but decided to forego his GP position in KPCB XI (which means that he is less involved in the fund, and the firm).

He sees opportunities in peer-to-peer infrastructure: communication, media distribution and mangling (as in being on the computer and im’ing whilst watching TV - something I do all the time), etc. He also states that personalization has not yet started to deliver on its promises on the web.

Vinod states that there is still too much capital in the coffins of the VCs, even if a lot of the overhang (‘leftovers’ from the capital raised by the bubble funds) has been largely used up. This makes raising money ‘too easy’, and cautions startups being funded nowadays that they should not take this as a sign of future success (neither on their capacity to execute nor raise further funds), and that management teams should spend frugally by trying things at small scale and get market response.

On the subject of search, he recalls Excite’s rebuttal of Vinod’s suggestion of buying Google’s technology for $1M (very very early on) because Excite guys thought they could do so much better. Lesson: be open to other’s capabilities to disrupt your turf - even when you are successful (actually, especially when you are successful because you might become complacent).

The discussion moves to the topic of trust in the blogosphere, put in the perspective of MSM and reporting on disasters, like Katrina and London bombings. Vinod argues that he will trust more aggregated reports from hundreds of bloggers rather than CNN, even if there is no blog trust/reputation infrastructure (I would argue that linking behavior is an OK proxy for now - Scoble would say that this is what we get with Memeorandum).

I don't think so re. Memeorandum (as I stated earlier, his is a genuine big picture thinker and thought leader, Memeorandum is a step in the right direction, but in no way the final destination envisaged).

On the mobile revolution, he sees an increased usage, way beyond ringtones and wallpapers, into communication and new types of interactions. An example he gives is using cell phones to help people improve their english (a real challenge for Chinese, Indian… and French people :-). Just in Shanghai, there are over 5M customers/users of such a product.

Moving to education, Vinod states that kids now need to learn about sifting through thousands and thousands of search engine results on any topic they might research, and critically assessing which source to trust or not. He would use new communcation mechanisms to offer remote tutoring, and would open source textbooks in a wikipedia model - which would save billions of dollars every year in California that could be reinvested into more critical projects.

Staying with open source, he gives the example of patented seeds that could benefit greatly to developing country and their ability to engineer seeds matching their particular requirements and environment. This is not feasilble today because of patent protection.

On improving on Google’s relevance (question from Michael Yang from Become.com), Vinod sees value in collaborative filtering, and diverse applications of information retrieval and extraction.

(Via Software Only.)

Key data points:

• Market cap of big 5: $2B (2000 pre-IPO), $178B (2000 peak), $32B (2002 trough) $261B (2005)
• 27% of US Internet users read blogs
• 54MM registered Skype users (9/05) - fastest product ramp ever?
• China - More Internet users < age of 30 than anywhere
• S. Korea Broadband penetration of 70%+ - No. 1 in world
• Mobile is most important direction now

Conclusion: first ten years (1995-2005) of commercial Internet were a warm up act for what is about to happen

"

(Via Silkworm Blog.)

Here is what my Virtuoso based blog system enables me to do with ease:

1. Grep all the URIs from Dare's "Mash-ups 2.0 - Where's the Business Model?" post via its linkblog.

2. Use Dare's Trip report to track broader Web 2.0 conversation across the blogosphere via its Summary Page.

Web 2.0 Conference Trip Report: Mash-ups 2.0 - Where's the Business Model?: "

I attended the panel on business models for mash-ups hosted by Dave McClure,
Jeffrey McManus, Paul Rademacher, and Adam Trachtenberg.

A mash up used to mean remixing two songs into something new and cool but now the term has been hijacked by geeks to means mixing two or more web-based data sources and/or services.

Paul Rademacher is the author of the Housing Maps mash-up which he used as a way to find a house using Craig'sList + Google Maps. The data obtained from Craig's List is fetched via screen scraping. Although Craig's List has RSS feeds, they didn't meet his needs. Paul also talked about some of the issues he had with building the site such as the fact that since most browsers block cross-site scripting using XMLHttpRequest then a server needs to be set up to aggregate the data instead of all the code running in the browser. The site has been very popular and has garnered over 900,000 unique visitors based solely on word-of-mouth.

The question was asked as to why he didn't make this a business but instead took a job at Google. He listed a number of very good reasons

1. He did not own the data that was powering the application.
2. The barrier to entry for such an application was low since there was no unique intellectual property or user interface design to his application

I asked whether he'd gotten any angry letters from the legal department at Craig's List and he said they seem to be tolerating him because he drives traffic to their site and caches a bunch of data on his servers so as not to hit their servers with a lot of traffic. 

A related mash-up site which scrapes real estate websites called Trulia was then demoed. A member of the audience asked whether Paul thought the complexity of mash-ups using more than two data sources and/or services increased in a linear or exponential fashion. Paul said he felt it increased in a linear fashion. This segued into a demo of SimplyHired with integrates with a number of sites including PayScale, LinkedIn, Job databases, etc.

At this point I asked whether they would have service providers giving their perspective on making money from mash-ups since they are the gating factor because they own the data and/or services mash-ups are built on. The reply was that the eBay & Yahoo folks would give their perspective later.

Then we get a demo of a Google Maps & eBay Motors mash-up. Unlike the Housing Maps mash-up, all the data is queried live instead of cached on the server. eBay has dozens of APis that encourage people to build against their platform and they have an affiliates program so people can make money from building on their API. We also got showed Unwired Buyer which is a site that enables you to bid on eBay using your cell phone and even calls you just before an auction is about to close. Adam Trachtenberg pointed out that since there is a Skype API perhaps some enterprising soul could mash-up eBay & Skype.

Jeffrey McManus of Yahoo! pointed out that you don't even need coding skills to build a Yahoo! Maps mash-up since all it takes is specifying your RSS feed with longitude and latitude elements on each item to have it embedded in the map. I asked why unlike Google Maps and MSN Virtual Earth, Yahoo! Maps doesn't allow users to host the maps on their page nor does there seem to be an avenue for revenue sharing with mash-up authors via syndicated advertising. The response I got was that they polled various developers and there wasn't significant interest in embedding the maps on developer's sites especially when this would require paying for hosting.

We then got showed a number mapping mashups including a mashup of the London bombings which used Google Maps, Flickr & RSS feeds of news (the presenter had the poor taste to point out opportunities to place ads on the site), a mashup from alkemis which mashes Google Maps, A9.com street level photos and traffic cams, and a mash-up from Analygis which integrates census data with Google Maps data.

The following items were then listed as the critical components of mash-ups
 - AJAX (Jeffrey McManus said it isn't key but a few of the guys on the panel felt that at least dynamic UIs are better)
 - APIs
 - Advertising
 - Payment
 - Identity/Acct mgmt
 - Mapping Services
 - Content Hosting
 - Other?

On the topic of identity and account management, the problem of how mash-ups handle user passwords came up as a problem. If a website is password protected then user's often have to enter their usernames and passwords into third party sites. An example of this was the fact that PayPal used to store lots of username/password information of eBay users which caused the company some consternation since eBay went through a lot of trouble to protect their sensitive data only to have a lot of it being stored on Paypal servers.

eBay's current solution is similar to that used by Microsoft Passport in that applications are expected to have user's login via the eBay website then the user is redirected to the originating website with a ticket indicating they have been authenticated. I pointed out that although this works fine for websites, it offers no solution for people trying to build desktop applications that are not browser based. The response I got indicated that eBay hasn't solved this problem.

My main comment about this panel is that it didn't meet expectations. I'd expected to hear a discussion about turning mashups [and maybe the web platforms they are built on] into money making businesses. What I got was a show-and-tell of various mapping mashups. Disappointing.

"

(Via Dare Obasanjo aka Carnage4Life.)

• Services, not packaged software, with cost-effective scalability
• Control over unique, hard-to-recreate data sources that get richer as more people use them
• Trusting users as co-developers
• Harnessing collective intelligence
• Leveraging the long tail through customer self-service
• Software above the level of a single device
• Lightweight user interfaces, development models, AND business models

• Exposes Web services that can be accessed on any device or platform by any developer or user. RSS feeds, RESTful APIs and SOAP APIs are all examples of Web services.
• Harnesses the collective knowledge of its user base to benefit users
• Leverages the long tail through customer self-service

Web 2.0 is a web of executable service invocation endpoints (those Web Services URIs) and well-formed content (all of that RSS, Atom, RDF, XHTML, etc. based Web Content out on the NET). The executable service invocation endpoints and well-formed content are accessible via URIs.

Put in even simpler terms, Web 2.0 is an incarnation of the web defined by URIs for invoking Web Services and/or consuming or syndicating well-formed content.

Looks like I've self edited my own definition in the process. :-)

If you don't grok this definition then consider using it as a trigger for taking a closer look at the dynamics that genuinely differentiate Web 1.0 and Web 2.0.

Web 1.0 was about web sites geared towards an interaction with human beings as opposed to computers. In a sense this mirrors the difference between HTML and XML.

A simple example (purchasing a book):

amazon.com provides value to you by enabling you to search and purchase the desired book online via the site http://www.amazon.com.

In the Web 1.0 era the process of searching for your desired book, and then eventually purchasing the book in question, required visible interaction with the site http://www.amazon.com. In today's Web 2.0 based Web the process of discovering a catalog of books, searching for your particular book of interest, and eventually purchasing the book, occurs via Web Services which amazon has chosen to expose via an executable endpoint (the Web point of presence for exposing its Web Services).

Direct interaction via http://www.amazon.com is no longer required. A weblog can quite easily associate keywords, tags, and post categories with items in amazon.com's catalogs. In addition, weblogs can also act as entry points for consuming the amazon.com value proposition (making books available for purchase online), by enabling you to purchase a book directly from the weblog (assuming the blog owner is an amazon associate etc..). Now compare the impact of this kind of value discovery and consumption cycle driven by software to the same process driven by humans interaction with a static or dynamic HTML page (Web 1.0 site).

To surmise, Web 2.0 is a reflection of the potential of XML expressed through the collective impact of Web Services (XML based distributed computing) and Well-formed Content (Blogosphere, Wikisphere, XHTML micro content etc.). The potential simply comes down to the ability to ultimately connect events, triggers, impulses (chatter, conversation, etc.), and data in general via URIs.

Let's never forget that XML is the reason why we have a blogosphere (RSS/Atom/RDF are applications of XML). Likewise, XML is also the reason why we have Web Services (doesn't matter what format).

As I have stated in the past, we must go by Web 2.0 en route what is popularly referred to as the Semantic Web (it will be known by another name by the time we get there; 3.0 or 4.0, who knows or cares?). At the current time, the prerequisite activity of self annotation is in full swing on the current Web, thanks to the inflective effects of Web 2.0.

BTW - Would this URI to all Semantic Web related posts on my blog pass the Web 2.0 litmus test? Likewise, this URI to all Web 2.0 related posts? I wonder :-)

Tim O'Reilly has posted a meme map of Web 2.0, from the 'What is Web 2.0?' brainstorming session at FOO Camp 2005. Hat-tip Josh for the link. It's kind of a business model map:

Click here for full image

The orange box in the middle and brown ovals at the bottom cover some the themes I'll be writing about in the next chapter of Josh and I's book. The chapter is tentatively titled 'Building a Web 2.0 Business' and will explore the principles of Web 2.0 business. e.g. 'Services, not packaged app'.

I've just finished my first chapter, which was a general introduction to the 'Web as Platform' concept. So this will be an interesting follow-on from that.

Alex Barnett has done a nice 'Microsoft mash-up', inserting links.

(Via Read/Write Web.)

A Webpage is Not An API or a Platform (The Populicio.us Remix): "

A few months ago in my post GMail Domain Change Exposes Bad Design and Poor Code, I wrote Repeat after me, a web page is not an API or a platform. It seems some people are still learning this lesson the hard way. In the post The danger of running a remix service Richard MacManus writes

Populicio.us was a service that used data from social bookmarking site del.icio.us, to create a site with enhanced statistics and a better variety of 'popular' links. However the Populicio.us service has just been taken off air, because its developer can no longer get the required information from del.icio.us. The developer of Populicio.us wrote:

'Del.icio.us doesn't serve its homepage as it did and I'm not able to get all needed data to continue Populicio.us. Right now Del.icio.us doesn't show all the bookmarked links in the homepage so there is no way I can generate real statistics.'

This plainly illustrates the danger for remix or mash-up service providers who rely on third party sites for their data. del.icio.us can not only giveth, it can taketh away.

It seems Richard Macmanus has missed the point. The issue isn't depending on a third party site for data. The problem is depending on screen scraping their HTML webpage. An API is a service contract which is unlikely to be broken without warning. A web page can change depending on the whims of the web master or graphic designer behind the site.

Versioning APIs is hard enough, let alone trying to figure out how to version an HTML website so screen scrapers are not broken. Web 2.0 isn't about screenscraping. Turning the Web into an online platform isn't about legitimizing bad practices from the early days of the Web. Screen scraping needs to die a horrible death. Web APIs and Web feeds are the way of the future.

(Via Dare Obasanjo aka Carnage4Life.)

This site could provide a great exposure point for some very old Web 2.0 (nee "Web Services") demos from our Virtuoso tutorials / demos site.

I ran into two work-related problems today that left me feeling like there are some aspects of two very recent (Web 2.0-esque if we wish to join the buzzword orgy of late) architectures (REST/Services and XForms) that are problematic:

Demonstrating an Achilles Heel Of XML Messaging

XML as a medium for remote communication (evangelized more with WSDL-related architectures than in REST) has over-stated its usefullness in at least one concrete regard, in my estimation. I've had a hard time taking most of the architectural arguments on the pros/cons of SOAP/XML-RPC versus REST seriously because it seems to be nothing more than buzzword warfare. However, I recently came across a concrete, real world example of the pitfalls of implementing certain remote service needs on XML-based communication mediums (such as SOAP/XML-RPC).

If the objects/resources you wish to manipulate at the service endpoints are run of the mill (consider the standard cliche purchase order example(s)) then the benefits of communicating in XML is obvious: portability, machine readability, extensibility, etc.. However consider the scenario (which I face) in which the objects/resources you wish to manipulate are XML documents themselves! This scenario seems to work to the disadvantage of the communication architecture.

Lets say you have a repository at one end (which I do) that has XML documents you wish to manipulate remotely. How do you update the documents? I've discussed this before (see: Base64 encoded XML content from an XForm) so I'll spare the details of the problem. However, I will mention that in retrospect this particular problem further emphasizes the advantage of a MinimalistRemoteProcedureCall (MRPC) approach - MRPC is my alternative acronym for REST :).

Consider the setContent message:

[SOAP:Envelope]
    [SOAP:Body]
     [foo:setContent]
       [path] .. path to document [/path]
       [src]... new document as a fragment ...[/src]
     [/foo:setContent]
   [/SOAP:Body]
[/SOAP:Envelope]

Notice that the location of the resource we wish to update is embedded within the message transmitted (via SOAP), which is transported on top of another communication medium (HTTP) that already has the neccessary semantics for saying the same thing:

Set the content of the resource identified by a path

In the SOAP scenario, the above message is delivered to a single service endpoint (which serves as an external gateway for all SOAP messages) which has to then parse the entire XML message in order to determine the method invoked (setContent in this case) and the parameters passed to it (both of which are only header information on a document that consists mostly of the new document).

However, in the MRPC scenario this service would be invoked simply as an HTTP PUT request sent directly to the XML document we wish to update:

Method: PUT
Protocol:  HTTP/1.0
URI: http://remoteHost:port/< .. path to XML document ..>
CONTENT:
... new document in it's entirety ..

Here, there is no need for a service middleman to interpret the service requested (and no need to parse a large XML document that contains another document embedded as a fragment). The HTTP request by itself specifies everything we need and does it using HTTP alone as the communication medium. This is even more advantageous when the endpoint is a repository that has a very well defined URI scheme or general addressing mechanism for it's resources (which 4Suite does, the repository in my case).

The Headaches of Base 64 Encoding in XForms

Since i didn't have the option of a REST-based service architecture (the preferred solution) I was relegated to having to base64 encode the new XML content and embed it within the XML message submitted to the service endpoint, like so:

[SOAP:Envelope]
   [SOAP:Body]
     [foo:setContent]
       [path] .. path to document [/path]
       [src]... base64 encoding of new document's serialization ...[/src]
     [/foo:setContent]
   [/SOAP:Body]
[/SOAP:Envelope]

Base 64 seemed like the obvious encoding mechanism mostly because it would seem from an interpretation of the XForms specification that due to the data binding restrictions of the Upload Control when bound to instances of type xsd:base64Binary a conforming XForms processor is responsible for having the capability to encode to Base 64 on the fly. Now, this is fine and dandy if the XML content you wish to submit is retrieved from a file on the local file system of the client communicating remotely with the server. However, what if you wish to use an instance (a live DOM) as the source for the update? This seems like a very reasonable requirement given that one of the primary motivation of XForms is to encourage the use of XML instances as the user interface data model (providing a complete solution to the 'M' in the MVC architecture.)

However:

• There is no mechanism within XForms for serialising live instances (there needs to be such a standard so implementations don't create their own proprietary mechanisms)
• There is no mechanism within XForms for explicitely encoding text in some portable binary format (which is incredibly useful IMHO - as shown above)

(Via Uche Ogbuji.)

While I'm still trying to figure this out, you should read Shelley's original post, Steve Levy, Dave Sifry, and NZ Bear: You are Hurting Us and see whether you think the arguments against blogrolls are as wrong as I think they are.

"..The Technorati Top 100 is too much like Google in that ‘noise’ becomes equated with ‘authority’. Rather than provide a method to expose new voices, your list becomes nothing more than a way for those on top to further cement their positions. More, it can be easily manipulated with just the release of a piece of software.."

Ajax, Hard Facts, Brass Tacks ... and Bad Slacks

A number of people have contacted me recently about Ajax [1] -- a catchy name -- coined to provide an umbrella term for a particular group of technologies used to build web applications. The use of the word comes from Jesse James Garrett in a recent blog [2], and describes a class of internet applications written using JavaScript in a browser. By using JavaScript these apps have full access to the DOM, and as a consequence are able to make all sorts of changes to the page that the user is interacting with, without having to go back to the server.

When the application does need to go back to the server -- to deliver some data and get a response -- the idea is to keep the DOM intact so that the user has a smooth experience. This means that all communication with the server needs to take place outside of the normal HTML form mechanism, since this would obviously replace the current page.

Ajax addressed this, with what it calls 'asynchronous-JavaScript' -- retrieve only the data you need, and then directly manipulate the DOM to get the effect you want. 'Asynchronous-JavaScript' accounts for the first few letters of the name, with the remainder being the obligatory 'X' for XML (although XML is not really key to this technology, and many of the applications that are often cited as Ajax-apps don't use XML as the data medium).

Buzzing

The response to Ajax has been pretty positive. In fact the only negatives have been either to suggest a change of name or to moan a little that "I've been doing this for years, why hasn't anyone noticed me?" (I won't put any links to those sort of articles, since they are a little embarassing -- after all, everyone has been doing this for years!)

Anyway, despite a couple of sour-pusses, the software community is almost universally excited, and the blog wires have glowed over the last few months with descriptions of Google Maps, GMail, and so on.

Just about everyone who has asked me about Ajax has expected me to be disappointed. Surely, they say, this makes the case for XForms weaker? But my answer is the exact opposite -- XForms and standards-based web applications are in every way superior to the techniques described as Ajax, since the whole raison d'être of XForms and XHTML 2 is to address the very problems that Ajax-like techniques suffer from.

That may come across as a little bold...so perhaps I should explain.

From Workaround to Feature

We've all been using HTML mark-up for years now, and the language hasn't changed much in that time. As a consequence, the increasing demand for more complex web-pages has meant that the balance in our documents has shifted increasingly from vanilla mark-up to 'the workaround'.

Whether it's providing tooltips, dynamic/repeating data sections, or small portions of our page that change without having to request a new document, we've generally had to dive into script. But the shift from mark-up to script has meant that the mark-up language itself has been relegated to a mere carrier for programs.

Unfortunately this means that no-one gains -- it's annoying for the programmer to have to produce ever more convoluted spaghetti JavaScript to meet the demands of their audience, but it's also annoying for the non-programmer, who probably only wants a tooltip. And its particularly annoying for those who want to use documents on the web for more ambitious applications to find that most of the important stuff in a document is hidden away in script.

All is not lost, however, since this collection of 'workarounds' provides a rich source of real-life patterns that appear for authors and programmers, time and again. They may be workarounds, but they are much-needed ones.

The aim of the new generation of languages like XForms and XHTML 2 is to take these 'common patterns' and turn them into mark-up. Just like the HTML elements <a> and <form> pack an enormous amount of functionality into deceptively simple tags, so too can new declarative mark-up capture patterns that have emerged 'in the wild'.

(Note that this is the opposite of so-called folksonomies, where popular practice that occurs in the wild is left it the wild, and codification is regarded as a dirty word.)

The XML HTTP Request Object

Let's take the much talked about XML HTTP Request Object (XMLHttpRequest). If you are not familiar with it, it was originally part of Microsoft's XML parser, and allows you to send and receive data outside of the normal HTML form processing. Since it's a handy feature to have in a client, other browsers have followed suit and it's now becoming the 'standard' way to communicate with servers without messing up your page. It's a corner-stone of Ajax. (A good summary with examples is on Jim Ley's jibbering.com site [3].)

But...we need to be clear that we're using XMLHttpRequest to get round a weakness in HTML forms. The problem we have is that even if you know that a server is about to give you some data, and the server knows it's about to give you some data, there's no way to tell your form that -- instead your page will be wiped out and replaced with whatever the server sends back.

Of course, constant round-tripping doesn't make it completely impossible to produce applications, and a lot of books and airline tickets are bought every day without the facility to get 'just the data'. But we all know it would reduce network traffic and create a smoother user experience if we could just send a list of books or seats, rather than a whole new page.

Over the years applications such as Microsoft's Outlook Web Access (OWA), have had to step around the HTML form to get just the data they need. But, whilst OWA considerably predates GMail, until the advent of XMLHttpRequest, the techniques used were quite difficult to manage. (Google Suggest is often cited as a good example of an Ajax-app, but interestingly merges old and new techniques; XMLHttpRequest is used to obtain a piece of JavaScript from a server, and this script contains a call to a client-side function, but using server-provided parameters. It's one of the techniques you might have used in the past with a hidden frame.)

So as many have said on their blogs, XMLHttpRequest is not a newly devised technique, but rather a generally accepted replacement for a very old technique. But ultimately that technique is a workaround since the real problem is that HTML forms will always replace the current page.

Beyond HTML Forms

Whilst XMLHttpRequest gives us a way to get data to and from the server without losing our document, we've unfortunately thrown the baby out with the bath-water; whatever the weaknesses of HTML forms, you have to acknowledge that they are pretty simple to use. Here's an abbreviated version of Google's search form (note that the mark-up is HTML, not XML):

<form action=/search name=f>
  <input type=hidden name=hl value=en>
  <input maxLength=256 size=55 name=q value="">
  <input type=submit value="Google Search" name=btnG>
</form>

As you can see, the simple problem with HTML forms is that we don't say anything about where the data should go when we've received it from the server. The assumption in HTML of old is that we are just doing a kind of 'super-navigation', and no matter what we send to the server, it will only ever give us back a new web-page. (To put it a different way, you could say that <a> and <form> are pretty much the same thing.)

To see how this problem is resolved, let's code the same Google search in XForms:

<xf:submission id="sub-search"
 action="http://www.google.com/complete/search?hl=en"
 method="get" separator="&"
 replace="all"
/>


<xf:input ref="q">
  <xf:label>Query:</xf:label>
</xf:input>


<xf:submit submission="sub-search">
  <xf:label>Google Search</xf:label>
</xf:submit>

Although it will do exactly the same -- right down to replacing the current page -- it's a little different to the HTML mark-up. But the changes in structure have given us some major benefits, from accessible labels on our form controls, to the possibility of many different submissions for the same data.

But what it has also given us is the possibility of solving our data update problem. The replace attribute is actually optional in XForms, but I showed it in the previous mark-up so that you can compare it to this:

<xf:submission id="sub-search"
 action="http://www.google.com/complete/search?hl=en"
 method="get" separator="&"
 replace="instance"
/>

In this example the data returned from the server will just replace the instance that was sent, and our page will remain completely intact. (The replace attribute can take the values all, instance, or none.)

I won't show the full equivalent using XMLHttpRequest since it's pretty large, but I'll give a flavour of it. (Jim Ley's page -- referenced earlier -- shows how to search Google with XMLHttpRequest.)

The Script Version

First we need to create an XMLHttpRequest object, but we need to do it in such a way that it will work on both Mozilla and IE:

var req;


function loadXMLDoc(url) {
    // native XMLHttpRequest object
    if (window.XMLHttpRequest) {
        req = new XMLHttpRequest();
        req.onreadystatechange = readyStateChange;
        req.open("GET", url, true);
        req.send(null);
    // IE/Windows ActiveX version
    } else if (window.ActiveXObject) {
        req = new ActiveXObject("Microsoft.XMLHTTP");
        if (req) {
            req.onreadystatechange = readyStateChange;
            req.open("GET", url, true);
            req.send();
        }
    }
}

When a document is loaded via this function, the readyStateChange() method is invoked:

function readyStateChange() {
    // '4' means document "loaded"
    if (req.readyState == 4) {
        // 200 means "OK"
        if (req.status == 200) {
            // do something here
        } else {
            // error processing here
        }
    }
}

From a programming point of view, I guess you could say that there isn't a lot wrong with this, but then from a programming point of view there wasn't a lot wrong with Z80 or 6502 assembly languages -- I just wouldn't want to go back to them!

But the most important issue is that we have lost the very thing that was responsible for HTML's success -- the use of simple, clear, declarative mark-up, in which we simply state our intent, without having to write a program to do it for us. After all, the web took off because authors only had to master <a> in order to enter the exciting new world of 'hypertext' -- but XMLHttpRequest raises the bar again, and takes us right back into the heart of geek-world.

Beyond XMLHttpRequest

But in keeping with the principle that I outlined above -- that XForms and XHTML 2 try to provide mark-up for commonly existing design patterns -- let's see if there are any other patterns that XMLHttpRequest has thrown up.

You will have noticed in the earlier script that we had tests for success and failure:

if (req.status == 200) {
  // do something here
} else {
  // error processing here
}

XForms provides the same functionality through the use of events -- on success do this, on failure do that. This is far more powerful, since it hides the protocol-specific aspects of this code ("200" may be 'success' for HTTP, but it isn't 'success' when saving data to the hard-drive or sending an email).

XForms uses declarative mark-up to express those events, which again dramatically reduces coding:

<xf:action ev:observer="sub-search" ev:event="xforms-submit-error">
  <xf:message level="modal">
    Submission failed
  </xf:message>
</xf:action>

But there's lots, lots more in the submission part of XForms:

• it can provide full XML Schema validation before submitting the data;


• there is built in support for numerous types of serialisation, such as multipart/related;


• abstract methods are used so the code is independent of protocol. For example, since put means the same thing whether the target URL begins http: or file:, a form with relative paths will run unchanged on a local machine or a web server;


• it's extensible -- in formsPlayer 2.0 we have used the submission element to read and write from an ADO database, allowing programmers to convert forms from using the web to using a local database by doing nothing more than changing a single target URL. (Try doing that with XMLHttpRequest!)

The submission part of XForms is in fact so powerful that it will eventually form a separate specification, for use in other languages.

From Patterns to Mark-up

And there are plenty more patterns out there that were crying out to be turned into mark-up, and which are now incorporated into XForms and XHTML 2. Do you remember the days when if we wanted a tooltip that contained mark-up -- perhaps an image, or bold text -- we had to use a carefully placed <div>, a CSS display: none;, a mouseover event handler and a timer? Nowadays the programmer with better things to do than work with spaghetti-JavaScript just uses the XForms <hint> element, and for free they get platform independence (and therefore accessibility), as well as the ability to insert any mark-up.

And what about the days when we had to write code to open up a text-to-speech engine, and then invoke the various methods on the object to get it to speak its mind? Nowadays who wouldn't just use a CSS property on their XForms' messages?

Bad Slacks

And do you remember...I'm sorry, this one always makes me laugh...do you remember how we used to write lots of JavaScript to recalculate the shopping-cart when a new item was added? I know it's hard to believe -- it's like looking at old photos of us all wearing flares. Anyway, thank God for straight trousers and the XForms dependency-engine.


But enough of the good old days, the days of assembly language, C and JavaScript...let's stick with the new.

Do Try This at Home

To round all of this off, we'll take a look at Google Suggest, and we'll use XForms to implement it. I'll walk through the demo in a separate blog [4] so that this one doesn't get too cluttered -- and hopefully by disecting this simple but useful application, we can show how declarative mark-up scores over scripting.


[1] Will AJAX help Google clean up?, c|net, http://news.com.com/Will+AJAX+help+Google+clean+up/2100-1032_3-5621010.html

[2] Ajax: A New Approach to Web Applications, Jesse James Garrett, Adaptive Path blog, http://www.adaptivepath.com/publications/essays/archives/000385.php

[3] Using the XML HTTP Request object, http://jibbering.com/2002/4/httprequest.html

[4] "Google Suggest" Using XForms, http://internet-apps.blogspot.com/2005/04/google-suggest-using-xforms.html

Tags: xforms | xbl | webapps | ajax | javascript

Advertising in RSS is just starting now, for all practical purposes. If we wanted to, as an industry, reject the idea, we could.

The Internet Archive initiative is building up an amazing collection of content that includes this "must watch" movie about the somewhat forgotten hypercard development environment.

As I watched the hypercard movie I obtained clear reassurance that my vision of Web 2.0 as critical infrastructure for a future Semantic Web isn't unfounded. The solution building methodology espoused by hypercard is exactly how Semantic Web applications will be built, and this will be done by orchestrating the componentary of Web 2.0.

When watching this clip make the following mental adjustments:

1. Swap hypercard stacks for discrete and/or composite services that have published endpoints exposed by Web 2.0 points of presence
   
   
2. Think of information taking the form of XML based content e.g. RSS, Atom, RDF, FOAF, XFN, and other future XML based data contextualization formats; all accessible via URIs
   
   
3. When the Apple Mac operating system is mentioned (or infered) think of the Internet (you don't need Windows, Mac OS, Linux, UNIX etc. to realize the vision, the network provided by the Internet is the Operating System)
   
   
4. When the Apple computer is mentioned simply think about a plethora of function specific devices (computers, mobile phones, PDAs etc.) that overtly or covertly provide conduits to the new operating environment (the Internet)
   
   
5. As you hear term "whole new body of people that are non programmers contributing there ideas" think about yourself and the increasing ease of participation that's beginning to take shape in this emerging frontier!
   
   
6. As for "Whole Earth Catalog", think Wikipedia or more recent efforts such as Answers.com.

Web 2.0 is a reflection of the web taking its first major step out of the technology stone age (certainly the case relative to the hypercard movie and "pre web" application development in general).

I absolutely understand the frustration expressed in Dare's post. An additional comment from my perspective is that this devolution has been in motion for a while and it is an integral part of the Misinformation and Disinformation based marketing strategies of many companies.

Misinformation and Disinformation only work when the target audience is apathetic (unfortunately the sad reality to date!). The bad news for marketing strategies that assume perpetuation of the aforementioned apathy is that the Internet is fundamentally reducing the cost of knowledge acquisition; by implication today's naive customer is tomorrow's knowledgeable decision maker. Vendors have a choice: build valuable products, and then market these products by disseminating knowledge. If a competitor's product is better than yours, get back to the labs (developers are actually stimulated and motivated by constructive challenges; especially as any developer worth his or her salt intrinsically believes they are the best at their craft deep down; and so they should!).

In the imminent future (Internet time) I expect to see the Wikisphere, Blogosphere, and other Web 2.0 (and beyond) realms bring clarity to the futility of Misinformation and Disinformation based marketing and PR (see my post about the Wikipedia induced inflection on Marketing and PR ).

BTW -- Does anyone know what's the difference between an ESB and a Universal Server? Likewise, the difference between a Virtual Database and an EII solution?

Today is one of those days where one topic appears to be on the mind of many across cyberspace. You guessed right! Its that Web 2.0 thing again.  

Paul Bausch brings Yahoo!'s most recent Web 2.0 contribution to our broader attention in this excerpt from his O'Reilly Network article:

I browse news, check stock prices, and get movie times with Yahoo! Even though I interact with Yahoo! technology on a regular basis, I've never thought of Yahoo! as a technology company. Now that Yahoo! has released a Web Services interface, my perception of them is changing. Suddenly having programmatic access to a good portion of their data has me seeing Yahoo! through the eyes of a developer rather than a user.

The great thing about this move by Yahoo! is two fold (IMHO):

1. It certainly makes Yahoo! a little more interesting of late. And it will certainly helps to distinguish Yahoo! from Google. Of course these companies overlap somewhat, but they are also pretty different in focus. I see Yahoo! increasingly as a portal platform play providing content access via syndication, publishing, and web services.
   
   
2. It will impact their bottom line pretty rapidly, and I hope they realize the impact of Web 2.0 when trying to explain the growth increments whenever they next report to their investors :-) In a previous post I expressed my sense of some confusion on the part of Jeff Bezos regarding the total contribution of AWS to Amazon's growth (BTW - my articles to date re. Amazon and Web 2.0 are available from here in a variety of XML syndication formats: Atom, RSS 2.0, RDF).
   

The great thing about the Platform oriented Web 2.0 is the ability to syndicate your value proposition (aka products and services) instead of pursuing fallable email campaigns. It enables the auto-discovery of products and services by user agents (the content aspect). Web 2.0 also provides an infrastructure for user agents to enter into a consumptive interactions with discrete or composite Web Services via published endpoints exposed by a platform (the execution aspect).

A scenario example:

You can obtain RSS feeds (electronic product catalogs) from Amazon today, although you have to explicitly locate these catalog-feeds since Amazon doesn't exploit feed auto-discovery within their domain.

If you use Firefox or another auto-discovery supporting RSS/Atom/RDF user agent; visit this URL; Firefox users should simply click on the little orange icon bottom right of the browser's window to its RSS feed auto-discovery in action.

Anyway, once you have the feeds the next step is execution endpoints discovery within the Amazon domain (the conduits to Amazon's order processing system in this example). At the current time there isn't broad standardization of Web Services auto-discovery but it's certainly coming; WSIL is a potential front runner for small scale discovery while UDDI provides a heavier duty equivalent for larger scale tasks that includes discovery and other related functionality realms.

Back to the example trail, by having the RSS/Atom/RDF feed data within the confines of a user agent (an Internet Application to be precise) nothing stops the extraction of key purchasing data from these feeds, plus your consumer data en route to assembling an execution message (as prescribed by the schema of the service in question)for Amazon's order processing/ shopping cart service.  All of this happens without ever seeing/eye-balling the Amazon site (a prerequisite of Web 1.0 hence the dated term: Web Site).

To summarize: Web 2.0 enables you to syndicate your value proposition and then have it consumed via Web Services, leveraging computer, as opposed to human interaction cycles. This is how I believe Web 2.0 will ultimately impact the growth rates (in most cases exponentially) of those companies that comprehend its potential. 

It is clear that in comparison to the Web of the last century, the nature of data on the Web later in this decade will be very different in the following aspects:

• Volume of data is growing by orders of magnitudes every year
  Multimedia and sensor data are becoming more and more common.
  
  
• Spatio-temporal attributes of data are important.
  
  
• Different data sources provide information to form the holistic picture.
  
  
• Users are not concerned with the location of data source, as long as its quality and credibility is assured. They want to know the result of the data assimilation (the big picture of the event).
  
  
• Real-time data processing is the only way to extract meaningful information
  Exploration, not querying, is the predominant mode of interaction, which makes context and state critical.
  
  
• The user is interested in experience and information, independent of the medium and the source.
  

Effectively, the nature of the knowledge on the Web is changing very fast. It used to be mostly static text documents; now it will be a combination of live and static multimedia, including text, data and documents with spatio-temporal attributes. Considering these changes, can the search engines developed for static text documents be able to deal with the needs of the Web? [via E M E R G I C . o r g]

No, but this doesn't render them useless since we wouldn't be at this point without the likes of Google, Yahoo! et al. But building upon the data substrate that web data oriented search engines provide is where the next batch of Information access and Knowledge discovery solutions will carve out their space. The symbiotic relationship between Google (data) and Gurunet's Answers.com (Information and Knowledge) is one interesting example.

The Web is a distributed collection of databases that implement variety of data storage models but are commonly accessible via protocols that rely on HTTP for transport (in-bound and out-bound messages) services. These databases increasingly using well-formed XML for query result (data contextualization) persistence and URIs for permenant reference. 'What Database?" you might ask, "What you once called your Web Site, Blog, Wiki, etc.." my time-less reply.

When you have the database that I describe above, and a collection of entry points from which discrete or composite Web Services can be invoked available from one or more internet domains, you end up with what I prefer to call "Web 2.0" presence, or what Richard McManus describes as: "The Web as a Platform".

Here is a collection of posts I have made in the past relating to Web 2.0, note that this list is dynamic since this blog is Virtuoso based (predictably):

Free Text Search with XHTML results page (with Virtuoso generated URIs for RSS, Atom, and RDF): http://www.openlinksw.com/blog/search.vspx?blogid=127&q=web+2.0&type=text&output=html 

It's also no secret that I believe that Virtuoso is a bleeding edge Web 2.0 technology platform (and more..). The URIs that I am exposing provide the foundation layer for other complimentary Web initiatives such as the Semantic Web (Web 2.0 provides infrastructure for the Semantic Web as time will show). They are also completely usable outside the realm of this blog.

BTW - Jon Udell is writing, experimenting with, and demonstrating similar concepts across feeds within his Web 2.0 domain.

These are indeed fun times!

Have RSS feeds killed the email star? silicon.com Feb 28 2005 12:58PM GMT

Business Week has a special report on Web services...
.......

That's a big difference. It means their business model can scale, and the bigger they get, the more profitable they become because they're building on that initial research and development investment." [via E M E R G I C . o r g]

On the issue of scale I would like to add (to the excerpt above) that fact that we will ultimately come to realize that Web Services facilitate computer cycle based value proposition consumption. This is exponentially greater that human interaction based value proposition consumption -- the fundamental difference between Web 1.0 and Web 2.0 IMHO.

It would be interesting to track the growth of companies in line with their adoption of Web Services based initiatives. I expect that when such a graph is produced in the future it will validate my point.

An immediate implication is that you can generate Google AdWords based adds using any development environment (Virtuoso's SQL Stored Procedure Language,  any .NET bound language, Java, C/C++, PHP, Ruby, Perl, Python, TCL etc.) that supports SOAP, WSDL, and I would presume WS-Security.

An even more interesting offshoot of this initiative from Google, is the fact that it could bring a degree of clarity to the issue of multi-protocol and multi-purpose servers (what I call Universal Servers e.g. OpenLink Virtuoso). For instance, you could manage AdWords campaigns across product portfolios using Triggers (the SQL database kind) or Notification Services.

Stickiness is a defining characteristic of Web 1.0 . It's all about eyeballs (site visitors) which implied ultimately that all early Web business models ended up down the advertising route.

I always felt that Web 1.0 was akin to having a crowd of people at your reception area seeking a look at your corporate brochures, and then someone realizes that you could start selling AD space in these brochures in response to the growing crowd size and frequency of congregation. The long-term folly of this approach is now obvious, as many organizations forgot their core value propositions (expressed via product offerings) in the process and wandered blindly down the AD model cul-de-sac, and we all know what happened down there..

Web 2.0 is taking shape (the inflection is in its latter stages), and the defining characteristics of Web 2.0 are:

1. Fabric of Executable Endpoints
   
2. Semantic Content (the RSS/RDF/Atom/FOAF semantic crumbs emerging from the Blogosphere are great examples of things to come re. XQuery queries over HTTP for instance) Migration from the Web Site (defined by static or dynamic HTML page generation) concept, to that of a "Web Point of Presence" (I don't know if this term will catch on, but the conceptual essence here is factual) that enables an organization to achieve the following:
   
• Package/catalog value proposition (product and services) using RSS/RDF/Atom
  
• Provide SOAP compliant Executable Endpoints (Web Services) for consuming value proposition (as opposed to being distracted by the AD model)
  
• Provide Web Services for consummating contracts associated with core value proposition Identification of internal efficiencies, new products/services that leverage Semantic Content and Web Services, and tangibly exploit:
  
• Composite Web Services construction from legacy monolithic application pools
  
• Standards based (e.g. BPEL) orchestration and integration of disparate composite services (across the Fabric referred to above)

When you factor in all of the above, the real question is whether Google and others are equipped to exploit Web 2.0?  To some degree, is the best answer at the current time as they have commenced the transition from "content only" web site to web platform (via the many Web Services initiatives that expose SOAP and REST interfaces to various services), but there is much more to this journey, and that's the devil in the "competitive landscape details".

From my obviously biased perspective, I think Virtuoso and Yukon+WinFS provide the server models for driving Web 2.0 points of presence (single server instances that implement multiple protocols). Thus, if Google, Yahoo! et al. aren't exploiting these or similar products, then they will be vulnerable over the long term to the competitve challenges that a Web 2.0 landscape will present.

Mozilla, FireFox, Opera et al. are all viable alternatives. Even better, get with the Web 2.0 program using the emerging pool of RSS Readers / Web Browser hybrids (note: unfortunately many sill use IE for browsing by default).

What really gets to me is that the fact that once the ill perceived destruction of Netscape was achieved, Microsoft went into predictable mode mode with IE (nothing to kill so why innovate, I mean we only innovate to kill products that potentially re-route users away from the Windows Lock-in / technology cul-de-sac etc..).

Internet Explorer Frame Injection Vulnerability “Mark Laurence has discovered a 6 year old vulnerability in Microsoft Internet Explorer, allowing malicious people to spoof the content of websites. The problem is that Internet Explorer doesn’t check if a target frame belongs to a website containing a malicious link, which therefore doesn’t prevent one browser window from loading content in a named frame in another window. Successful exploitation allows a malicious website to load arbitrary content in an arbitrary frame in another browser window owned by e.g. a trusted site. Secunia has constructed a test, which can be used to check if your browser is affected by this issue. This vulnerability is similar to an old vulnerability fixed by MS98-020 in Internet Explorer version 3 and 4. The vulnerability has been confirmed in a fully patched Internet Explorer 6 running on Microsoft Windows XP. Other versions of Internet Explorer may also be affected. Solution: Disable the following security setting: ‘Navigate sub-frames across different domains’. [Tools/Internet Options/Security tab in an Internet Explorer windows or Internet Options/Security tab from Control Panel.] Do not visit or follow links from untrusted websites.”

Web E-Mail: The New Hard Disk

..

If this "Internet Operating System" and Web 2.0 stuff is really happening, I think I've just found the filesystem we'll all be using--in one form or another.

Tim Berners-Lee provided a keynote at WWW2004 earlier this week, and Paul Ford provided a keynote breakdown from which I have scrapped a poignant excerpt that helps me illuminate Virtuoso's role in the inevitable semantic web.

First off, I see the Semantic Web as a core component of Web 2.x (a minor upgrade of Web 2.0), and I see Virtuoso as a definitive Web 2.0 (and beyond) technology, hence the use today of the branding term "Universal Server". A term that I expect to become a common product moniker in the not too distant future.

The first challenge that confronts the semantic web is the creation of Semantic content. How will the content be created? Ideally, this should come from data, at the end of the day this is a data contextualization process. The excerpt below from Paul's article highlights the point:

Rather than concerning themselves unduly with hewing to existing ontologies, Berners-Lee pushed developers to start using RDF and triples more aggressively. In particular, he wants to see existing databases exported as RDF, with ontologies created ad-hoc to match the structure of that data. Rather than using PHP scripts only to produce HTML, he suggested, create RDF as well. Then, when all of the RDF is aggregated, apply rules and see what happens. "Let's not fall back on handmade markup."

Data in existing databases does not have to be exported as RDF, especially if sensitivity to change is a specific contextual requirement. Naturally, the assumption is made that most databases don't have the ability to produce RDF so an additonal tool would be required to perform the data exports and transformation, and then a separate HTTP server makes this repurposed RDF data accessible over HTTP.

Later in the talk, he described a cascade of Semantic Web connections, postulating that one day, individuals may be able to follow links from a parts catalog to order status, from location to weather to taxes.

The final excerpt (above) outlines the kinds of interactions that the Semantic Web facilitates. The traversal from a "part catalog" to "order status", or from "location" to "weather" to "taxes", illustrates the roles that services and service orchestration will also play in the Semantic Web era.

Thus, we can safely deduce the following about the semantic web:

1. It has RDF at its foundation
2. We need to transform existing data into RDF; ideally retaining sensitivity to changes
3. Allows ontologies to be associated with RDF post generation
4. RDF graph navigation will be event driven and orchestrated (the cascading effect)
5. There will be an RDF Query Language (there are several burgeoning ones currently)
6. HTTP will be the prime transport protocol

I would also like to conclude that what we know today, as the monolithic "point of presence" on the web called a "Web Site" (which infers browsing and page serving), is naturally going to morph into a different kind of "point of presence" that is capable of delivering the following from a single process:

1. Serve up Semantic Data from existing data sources
2. Provide execution endpoints for Web Services
3. Provide an instigation point for events that trigger Service Orchestratio

This is what Virtuoso is all about, and why it is described as a "Universal Server"; a server instance that speaks many protocols, delivering a plethora of functionality (Database, Web Services Platform, Orchestration Engine, and more).

RSS information routers will emerge in 2004 with the following characteristics:

? Persistent storage of XHTML full-text/graphics/audio/video of RSS feeds
? XPATH search across local and Net stores
? Self-forming and reordering subscriptions lists based on the aggregated priorities of user-chosen domain experts
? Use of IM notification for post notification to aggregate affinity groups and active conversations
? Integration of Hydra-like collaborative tools for multi-author conference transcripts
? Videoconferencing routing and broadcast/recording tools
? Integration of speech recognition and real-time indexing to allow quoting of linear audio and video streams
? Mesh networked peer-to-peer synchronization engine for item propagation across shared spaces on multiple clients, including phones; iPods; and eventually Longhorn PDAs (circa 2006).

Armed with these tools, new industries will emerge in rapid succession:

? Metadata-driven directories that dynamically create RSS feeds based on affinity
? Virtual conferences
? IM/RSS presence networks for rich collaboration and e-mail replacement
? Content-generation tools based on small, routable XHTML objects
? A DRM network with enough creative and hardware support to blunt the Microsoft/RIAA DRM threat to peer-to-peer port hijacking.

It starts here with one of many excerpts from Scoble's blog:

Dave Winer, Jon Udell, and now Gerald Bauer says that Microsoft is killing the Web. Or trying to.

The guys above are pretty seasoned individuals (they save me a lot of writing too amongst other things).

Now here is a response from Microsoft?s Blog evangelist supremo Scoble to their comments and genuine concerns.

OK, let's assume that's true.

Microsoft has 55,000 employees. $50 billion or so in the bank.

Yet what has gotten me to use the Web less and less lately? RSS 2.0.

Seriously. I rarely use the browser anymore (except to post my weblog since I use Radio UserLand).

See the irony there? Dave Winer (who at minimum popularized RSS 2.0) has done more to get me to move away from the Web than a huge international corporation that's supposedly focused on killing the Web.

Now, let's look at what's really going on here. We're going back to being a great platform company. We're trying to provide a platform that lets developers build new applications that are impossible to build on other platforms. At the PDC you saw some of that. New kinds of forms. New kinds of games. New kinds of business apps. New kinds of experiences.

But, we also are looking for ways to make the Web better too. Now, we haven't talked about what we're doing with the browser. I hear that'll come later. Astute Longhorn testers have already seen that we snuck a pop-up ad blocker into the browser without telling anyone about it. Whoa. That means we're gonna turn off MSN's capabilities of selling popup ads.

I hear there's more coming too. But, why should we do it all? Wasn't the point of the past four years to get Microsoft to stop trying to do it all? The DOJ and now the European Union are still after us cause we tried to do it all. Instead, let's just go back and be a great platform company.

We just gave you a great foundation for a killer new kind of application. One that goes FAR beyond HTML. And, even if you stick with Mozilla, your experiences on Longhorn will get better. For instance, fonts are being rendered in the GPU now on Longhorn. Your Web pages will look better and behave better on Longhorn than they will on any other platform. Period.

And wait until Mozilla's and other developers start exploiting things like WinFS to give you new features that display Internet-based information in whole new ways.

If Microsoft really wants to create a better platform shouldn?t this be truly futuristic? If so, then it should issue the first major salvo by dropping the restrictions on Rotor?

We are moving into the distributed component based computing age where runtime environments (.NET CLR, Mono, J2EE, and others) act a Component Execution Junction boxes (instead of the Monolithic Operating Systems of today) in a continuum of services orchestrated by messages in response to events emanating from value consumption requests (what we call application behviour today) from a myriad of value consumers (application users).

There is no need for covert and protracted protection of an obsolete Windows Operating System (the underlying fear that keeps Rotor shackled in my opinion), since its obsolescence is in full motion as Longhorn clearly demonstrates.

Imagine a fusion of sorts across Microsoft .NET, Mono, and Rotor, with a single portable runtime as the end product (slotting nicely into its place in the imminent distributed component and services era). All the benefits of programming language independence in true glory - the ECMA-CLI is all about programming language independence. Now that would be unequivocally revolutionary, and Microsoft would actually be doing what I think it has been desperately trying to achieve for a long time; the delivery of really cool technology that seriously impact us all in a positive way without the usual World Domination Concerns. 

Anyway, back to the current reality where we have covert attempts to lock us all into Windows getting more and more transparent per technology release cycle. The very antithesis of what I espoused in the last paragraph (or dream). I believe that Scoble's instincts lie in this realm too, and you never know this evangelist may turn Messiah :-)

Here's the final excerpt from Scoble?s post:

There's a whole lot of more useful stuff coming. Both for the Web and for newer Internet-centric rich-client approaches. Personally, it's about time. I'm already using the Web less and less thanks to things like RSS 2.0.

I'm watching 636 sites every day. Try to do THAT in your Web browser.

So, yes, blame it on me. I'm trying to kill the Web. Isn't it time to move on? Didn't we move on from the Apple II? Didn't we move on from DOS? Didn't we move on from Windows 3.11? Can't you see a day when we move on from the Web and get something even more fantastic? I can. Dave Winer can. Why not you? [via The Scobleizer Weblog]

If you kill the Web en route to getting us a Portable Execution Junction box from Microsoft, I think you would have served mankind pretty damned well. We won't have to gripe about Web 1.0 (Browser Driven Web) because we would be well into Web 2.0 and beyond (which doesn?t define the Web experience predominantly via browsing).

Q: Amazon.com now runs sites and on-line operations for retailers such as Target and Toys 'R' Us. What's the future for that services business? A: It's a rapidly growing part of our business. And that goes from [large] companies that are customers of that all the way down to individuals using our Web services to tap into the fundamental platform that is Amazon.com. They can build their own applications very effectively. It's almost closer to an ecosystem. Q: So Amazon is becoming a kind of software platform a bit like Microsoft (MSFT )? A: People are building stuff that surprises us. That's what's so interesting about this. We've built this big base of technology to serve ourselves, and now we're opening it up and letting people access it. They're taking these fundamental pieces and building completely new things that not only would we have never gotten around to but in some cases maybe never even have thought of. There are thousands of developers who are building applications using Amazon Web services. The sky's the limit on their creativity. Q: What arises from all those efforts? A: People will be able to build very powerful applications by hooking together a whole bunch of Web services from a whole bunch of different companies. Q: What benefit is Amazon.com getting from this? A: It's too early to say. It's certainly not a major source of revenue for us. But when people use our Web services, they give us credit for that. That turns out to be very helpful.

In the year 2000 the question of the shape and form of XML data was unclear to many, and reading the article below basically took me back in time to when we released Virtuoso 2.0 (we are now at release 3.0 commercially with a 3.2 beta dropping any minute).

RSS is a great XML application, and it does a great job of demonstrating how XML --the new data access foundation layer-- will galvanize the next generation Web (I refer to this as Web 2.0.).

RSS: INJAN (It's not just about news)

RSS is not just about news, according to Ian Davis on rss-dev.
He presents a nice list of alternatives, which I reproduce here (and to which I�d add, of course, bibliography management)

• Sitemaps: one of the S�s in RSS stands for summary. A sitemap is a summary of the content on a site, the items are pages or content areas. This is clearly a non-chronological ordering of items. Is a hierarchy of RSS sitemaps implied here � how would the linking between them work? How hard would it be to hack a web browser to pick up the RSS sitemap and display it in a sidebar when you visit the site?
• Small ads: also known as classifieds. These expire so there�s some kind of dynamic going on here but the ordering of items isn�t necessarily chronological. How to describe the location of the seller, or the condition of the item or even the price. Not every ad is selling something � perhaps it�s to rent out a room.
• Personals: similar model to the small ads. No prices though (I hope). Comes with a ready made vocabulary of terms that could be converted to an RDF schema. Probably should do that just for the hell of it anyway � gsoh
• Weather reports: how about a week�s worth of weather in an RSS channel. If an item is dated in the future, should an aggregator display it before time? Alternate representations include maps of temperature and pressure etc.
• Auctions: again, related to small ads, but these are much more time limited since there is a hard cutoff after which the auction is closed. The sequence of bids could be interesting � would it make sense to thread them like a discussion so you can see the tactics?
• TV listings: this is definitely chronological but with a twist � the items have durations. They also have other metadata such as cast lists, classification ratings, widescreen, stereo, program type. Some types have additional information such as director and production year.
• Top ten listings: top ten singles, books, dvds, richest people, ugliest, rear of the year etc. Not chronological, but has definate order. May update from day to day or even more often.
• Sales reporting: imagine if every department of a company reported their sales figures via RSS. Then the divisions aggregate the departmental figures and republish to the regional offices, who aggregate and add value up the chain. The chairman of the company subscribes to one super-aggregate feed.
• Membership lists / buddy lists: could I publish my buddy list from Jabber or other instant messengers? Maybe as an interchange format or perhaps could be used to look for shared contacts. Lots of potential overlap with FOAF here.
• Mailing lists: or in fact any messaging system such as usenet. There are some efforts at doing this already (e.g. yahoogroups) but we need more information � threads; references; headers; links into archives.
• Price lists / inventory: the items here are products or services. No particular ordering but it�d be nice to be able to subscribe to a catalog of products and prices from a company. The aggregator should be able to pick out price rises or bargains given enough history.
[via Semantic Blogging Demonstrator]

Thus, if we can comprehend RSS (the blog article below does a great job) we should be able to see the fundamental challenges that are before any organization seeking to exploit the potential of the imminent Web 2.0 inflection; how will you cost-effectively create XML data from existing data sources? Without upgrading or switching database engines, operating systems, programming languages? Put differently how can you exploit this phenomenon without losing your ever dwindling technology choices (believe me choices are dwindling fast but most are oblivious to this fact).