1. You encounter DBpedia or the LOD Cloud Pictorial.
2. You look around (typically following your nose from link to link).
3. You attempt to publish your own stuff.
4. You get stuck.

The problems typically take the following form:

1. Functionality confusion about the complementary Name and Address functionality of a single URI abstraction
2. Terminology confusion due to conflation and over-loading of terms such as Resource, URL, Representation, Document, etc.
3. Inability to find robust tools with which to generate Linked Data from existing data sources such as relational databases, CSV files, XML, Web Services, etc.

To start addressing these problems, here is a simple guide for generating and publishing Linked Data using Virtuoso.

Step 1 - RDF Data Generation

Existing RDF data can be added to the Virtuoso RDF Quad Store via a variety of built-in data loader utilities.

Many options allow you to easily and quickly generate RDF data from other data sources:

• Install the Sponger Bookmarklet for the URIBurner service. Bind this to your own SPARQL-compliant backend RDF database (in this scenario, your local Virtuoso instance), and then Sponge some HTTP-accessible resources.
• Convert relational DBMS data to RDF using the Virtuoso RDF Views Wizard.
• Starting with CSV files, you can
  • Place them at an HTTP-accessible location, and use the Virtuoso Sponger to convert them to RDF or;
  • Use the CVS import feature to import their content into Virtuoso's relational data engine; then use the built-in RDF Views Wizard as with other RDBMS data.
• Starting from XML files, you can
  • Use Virtuoso's inbuilt XSLT-Processor for manual XML to RDF/XML transformation or;
  • Leverage the Sponger Cartridge for GRDDL, if there is a transformation service associated with your XML data source, or;
  • Let the Sponger analyze the XML data source and make a best-effort transformation to RDF.

Step 2 - Linked Data Deployment

Install the Faceted Browser VAD package (fct_dav.vad) which delivers the following:

1. Faceted Browser Engine UI
2. Dynamic Hypermedia Resource Generator
   • delivers descriptor resources for every entity (data object) in the Native or Virtual Quad Stores
   • supports a broad array of output formats, including HTML+RDFa, RDF/XML, N3/Turtle, NTriples, RDF-JSON, OData+Atom, and OData+JSON.

Step 3 - Linked Data Consumption & Exploitation

Three simple steps allow you, your enterprise, and your customers to consume and exploit your newly deployed Linked Data --

1. Load a page like this in your browser: http://<cname>[:<port>]/describe/?uri=<entity-uri>
   • <cname>[:<port>] gets replaced by the host and port of your Virtuoso instance
   • <entity-uri> gets replaced by the URI you want to see described -- for instance, the URI of one of the resources you let the Sponger handle.
2. Follow the links presented in the descriptor page.
3. If you ever see a blank page with a hyperlink subject name in the About: section at the top of the page, simply add the parameter "&sp=1" to the URL in the browser's Address box, and hit [ENTER]. This will result in an "on the fly" resource retrieval, transformation, and descriptor page generation.
4. Use the navigator controls to page up and down the data associated with the "in scope" resource descriptor.

Related

• Sample Descriptor Page (what you see post completion of the steps in this post)
• What is Linked Data, really?
• Painless Linked Data Generation via URIBurner
• How To Load RDF Data Into Virtuoso (various methods)
• Virtuoso Bulk Loader Script for RDF
• Bulk Loader Script for CSV
• Wizard based generation of RDF based Linked Data from ODBC accessible Relational Databases

Naturally, I am not expecting everyone to agree with me. I am simply making my contribution to what will remain facinating discourse for a long time to come :-)

Related

• Web 3.0 The Best Official Definition Imaginable -- Nova Spivack's

A pre-installed edition of Virtuoso for Amazon's EC2 Cloud platform.

What does it offer?

1. Low cost entry point to a game-changing Web 3.0+ (and beyond) platform that combines SQL, RDF, XML, and Web Services functionality
2. Flexible variable cost model (courtesy of EC2 DevPay) tightly bound to revenue generated by your services
3. Delivers federated and/or centralized model flexibility for you SaaS based solutions
4. Simple entry point for developing and deploying sophisticated database driven applications (SQL or RDF Linked Data Web oriented)
5. Complete framework for exploiting OpenID, OAuth (including Role enhancements) that simplifies exploitation of these vital Identity and Data Access technologies
6. Easily implement RDF Linked Data based Mail, Blogging, Wikis, Bookmarks, Calendaring, Discussion Forums, Tagging, Social-Networking as Data Space (data containers) features of your application or service offering
7. Instant alleviation of challenges (e.g. service costs and agility) associated with Data Portability and Open Data Access across Web 2.0 data silos
8. LDAP integration for Intranet / Extranet style applications.

From the DBMS engine perspective it provides you with one or more pre-configured instances of Virtuoso that enable immediate exploitation of the following services:

1. RDF Database (a Quad Store with SPARQL & SPARUL Language & Protocol support)
2. SQL Database (with ODBC, JDBC, OLE-DB, ADO.NET, and XMLA driver access)
3. XML Database (XML Schema, XQuery/Xpath, XSLT, Full Text Indexing)
4. Full Text Indexing.

From a Middleware perspective it provides:

1. RDF Views (Wrappers / Semantic Covers) over SQL, XML, and other data sources accessible via SOAP or REST style Web Services
2. Sponger Service for converting non RDF information resources into RDF Linked Data "on the fly" via a large collection of pre-installed RDFizer Cartridges.

From the Web Server Platform perspective it provides an alternative to LAMP stack components such as MySQL and Apace by offering

1. HTTP Web Server
2. WebDAV Server
3. Web Application Server (includes PHP runtime hosting)
4. SOAP or REST style Web Services Deployment
5. RDF Linked Data Deployment
6. SPARQL (SPARQL Query Language) and SPARUL (SPARQL Update Language) endpoints
7. Virtuoso Hosted PHP packages for MediaWiki, Drupal, Wordpress, and phpBB3 (just install the relevant Virtuoso Distro. Package).

From the general System Administrator's perspective it provides:

1. Online Backups (Backup Set dispatched to S3 buckets, FTP, or HTTP/WebDAV server locations)
2. Synchronized Incremental Backups to Backup Set locations
3. Backup Restore from Backup Set location (without exiting to EC2 shell).

Higher level user oriented offerings include:

1. OpenLink Data Explorer front-end for exploring the burgeoning Linked Data Web
2. Ajax based SPARQL Query Builder (iSPARQL) that enables SPARQL Query construction by Example
3. Ajax based SQL Query Builder (QBE) that enables SQL Query construction by Example.

For Web 2.0 / 3.0 users, developers, and entrepreneurs it offers it includes Distributed Collaboration Tools & Social Media realm functionality courtesy of ODS that includes:

1. Point of presence on the Linked Data Web that meshes your Identity and your Data via URIs
2. System generated Social Network Profile & Contact Data via FOAF?
3. System generated SIOC (Semantically Interconnected Online Community) Data Space (that includes a Social Graph) exposing all your Web data in RDF Linked Data form
4. System generated OpenID and automatic integration with FOAF
5. Transparent Data Integration across Facebook, Digg, LinkedIn, FriendFeed, Twitter, and any other Web 2.0 data space equipped with RSS / Atom support and/or REST style Web Services
6. In-built support for SyncML which enables data synchronization with Mobile Phones.

How Do I Get Going with It?

• Standard Installation Guide
• Personal or Service Specific DBpedia Installation Guide

First off, I am going to focus on the Semantic Data Web aspect of the overall Semantic Web vision (a continuum) as this is what we have now. I am also writing this post as a deliberate contribution to the discourse swirling around the real topic: Semantic Web Value Proposition.

Situation Analysis

We are in the early stages of the long anticipated Knowledge Economy. That being the case, it would be safe to assume that information access, processing, and dissemination are of utmost importance to individuals and organizations alike. You don't produce knowledge in a vacum! Likewise, you can produce Information in a vacum, you need Data.

The Semantic Data Web's value to Individuals

Increasingly, Blogs, Wikis, Shared Bookmarks, Photo Galleries, Discussion Forums, Shared Calendars and the like, have become invaluable tools for individual and organizational participation in Web enabled global discourse (where a lot of knowledge is discovered). These tools, are typically associated with Web 2.0, implying Read-Write access via Web Services, centralized application hosting, and data lock-in (silos).

The reality expressed above is a recipe for "Information Overload" and complete annihilation of ones effective pursuit and exploitation of knowledge due "Time Scarcity" (note: disconnecting is not an option). Information abundance is inversely related to available processing time (for humans in particular). In my case for instance, I was actively subscribed to over 500+ RSS feeds in 2003. As of today, I've simply stopped counting, and that's just my Weblog Data Space. Then add to that, all of the Discussions I track across Blogs, wikis, message boards, mailing lists, traditional usnet discussion forumns, and the like, and I think you get the picture.

Beyond information overload, Web 2.0 data is "Semi-Structured" by way of it's dominant data containers ((X)HTML, RSS, Atom documents and data streams etc.) lacking semantics that formally expose individual data items as distinct entities, endowed with unambiguous naming / identification, descriptive attributes (a type of property/predicate), and relationships (a type of property/predicate).

Devise a standard for Structured Data Semantics that is compatible with the Web Information BUS.

Produce structured data (entities, entity types, entity relationships) from Web 1.0 and Web 2.0 resources that already exists on the Web such that individual entities, their attributes, and relationships are accessible and discernible to software agents (machines).

Once the entities are individually exposed, the next requirement is a mechanism for selective access to these entities i.e. a query language.

Semantic Data Web Technologies that facilitate the solution described above include:

RDF - Data Model for structured data

RDF/XML - A serialization format for RDF based structured data

N3 / Turtle - more human friendly serialization formats for RDF based structured data

GRDDL - enables association between XHTML pages and XSLT stylesheets that facilitates loosely coupled "on the fly" extraction of RDF from non RDF documents

RDFa - enables document publishers or viewers (i.e those repurposing or annotating) to embed structured data into existing XHTML documents

eRDF - another option for embedding structured RDF data within (X)HTML documents

RDF Middleware - typically incorporating GRDDL, RDFa, eRDF, and custom extraction and mapping as part of a structured data production pipeline

Use of URIs or IRIs for uniquely identifying physical (HTML Documents, Image Files, Multimedia Files etc..) and abstract (People, Places, Music, and other abstract things).

SPARQL Query Language - the SQL analog of the Semantic Data Web that enables query constructs that target named entities, entity attributes, and entity relationships

SPARQL Protocol - a REST or SOAP style Web Service for transporting SPARQL Queries to Structured Data Sources.

SPARQL Results Serialization Formats - query results serialization formats that includes XML(sparql+xml) and JSON.

The Semantic Data Web's value to Organizations

Organizations are rife with a plethora of business systems that are built atop a myriad of database engines, sourced from a variety of DBMS vendors. A typical organization would have a different database engine, from a specific DBMS vendor, underlying critical business applications such as: Human Resource Management (HR), Customer Relationship Management (CRM), Accounting, Supply Chain Management etc. In a nutshell, you have DBMS Engines, and DBMS Schema heterogeneity permeating the IT infrastructure of organizations on a global scale, making Data & Information Integration the biggest headache across all IT driven organizations.

Alleviation of the pain (costs) associated with Data & Information Integration.

A dexterous data model (RDF) that enables the construction of conceptual views of disparate data sources across an organization based on existing web architecture components such as HTTP and URIs.

Existing middleware solutions that facilitate the exposure of SQL DBMS data as RDF based Structured Data include:

Virtuoso's Meta Schema Language for RDF Views of SQL Data (also see the Virtuoso SQL-RDF Technical White Paper)

D2RQ

DataGrid

Others

BTW - There is an upcoming W3C Workshop covering the integration of SQL and RDF data.

Conclusion

The Semantic Data Web is here, it's value delivery vehicle is the URI. The URI is a conduit to Interlinked Structured Data (RDF based Linked Data) derived from existing data sources on the World Wide Web alongside data continuously injected into the Web by organizations world wide. Ironically, the Semantic Data Web only platform that crystallizes the: Information at Your Fingertips vision, without development environment, operating system, application, or database lock-in. You simply click on a Linked Data URI and the serendipitous exploration and discovery of data commences.

The unobtrusive emergence of the Semantic Data Web is a reflection of the soundness of the underlying Semantic Web vision.

If you are excited about Mash-ups then your are a Semantic Web enthusiast and benefactor in the making, because you only "Mash" (brute force data extraction and interlinking) because you can't "Mesh" (natural data extraction and interlinking). Likewise, if you are a social-networking, open social-graph, or portable social-network enthusiast, then you are also a Semantic Data Web benefactor and enthusiasts, because your "values" (yes, the values associated with the properties that define you e.g your interests etc) are the fundamental basis for portable, open, social-networking, which is what the Semantic Data Web hands to you on a platter without compromise (i.e. data lock-in or loss of data ownership).

Read/WriteWeb via the OpenLink Data Web Browser (click on the different viewing tabs to see what structured data exploitation in action)

Read/WriteWeb via the Zitgist Data Web Browser

DBpedia (*note: I deliberately use DBpedia URIs in my posts where I would otherwise have used a Wikipedia article URI*)

Zitgist zLinks - Mike Bergman's Blog Post also demonstrating zLinks

New Semantic Data Web related features and enhancements include:

* A Javascript-based Fresnel processor enabling declarative RDF-based display templates for RDF Data Sources

* An XSLT template for generating HTML pages from the Fresnel processor's XML output

* Interactive Query Builder for SPARQL (iSPARQL). This version of the iSPARQL application includes support for INSERTs and DELETEs

* Enhanced Javascript-based N3/Turtle parser

* New Navigator viewer panel for RDF Browser.

*Project Home Page

*Source Code

*Live Features Demonstrations.

A vital component of the new Virtuoso release is the finalization of our SQL to RDF mapping functionality -- enabling the declarative mapping of SQL Data to RDF. Additional technical insight covering other new features (delivered and pending) is provided by Orri Erling, as part of a series of post-Banff posts.

Why is SQL to RDF Mapping a Big Deal?

A majority of the world's data (especially in the enterprise realm) resides in SQL Databases. In addition, Open Access to the data residing in said databases remains the biggest challenge to enterprises for the following reasons:

1. SQL Data Sources are inherently heterogeneous because they are acquired with business applications that are in many cases inextricably bound to a particular DBMS engine
2. Data is predictably dirty
3. DBMS vendors ultimately hold the data captive and have traditionally resisted data access standards such as ODBC (*trust me they have, just look at the unprecedented bad press associated with ODBC the only truly platform independent data access API. Then look at how this bad press arose..*)

Enterprises have known from the beginning of modern corporate times that data access, discovery, and manipulation capabilities are inextricably linked to the "Real-time Enterprise" nirvana (hence my use of 0.0 before this becomes 3.0).

In my experience, as someone whose operated in the data access and data integration realms since the late '80s, I've painfully observed enterprises pursue, but unsuccessfully attain, full control over enterprise data (the prized asset of any organization) such that data-, information-, knowledge-workers are just a click away from commencing coherent platform and database independent data drill-downs and/or discovery that transcend intranet, internet, and extranet boundaries -- serendipitous interaction with relevant data, without compromise!

Okay, situation analysis done, we move on..

At our most recent (12th June) monthly Semantic Web Gathering, I unveiled to TimBL and a host of other attendees a simple, but powerful, demonstration of how Linked Data, as an aspect of the Semantic Data Web, can be applied to enterprise data integration challenges.

Actual SQL to RDF Mapping Demo / Experiment

Hypothesis

Apparatus

Experiment / Demo

1. Declaratively map the Northwind SQL Schema to RDF using the Virtuoso Meta Schema Language (see: Virtuoso PL based Northwind_SQL_RDF script)
2. Start browsing the data by clicking on the URIs that represent the RDF Data Model Entities resulting from the SQL to RDF Mapping

Observations

1. Via a single Data Link click I was able to obtain specific information about the Customer represented by the URI "ALFKI" (act of URI Dereferencing as you would an Object ID in an Object or Object-Relational Database)
2. Via a Dynamic Data Page I was able to explore all the entity relationships or specific entity data (i.e Exploratory or Entity specific dereferencing) in the Northwind Data Space
3. I was able to perform similar exploration (as per item 2) using our OpenLink Browser.

Conclusions

The vision of data, information, or knowledge at your fingertips is nigh! Thanks to the infrastructure provided by the Semantic Data Web (URIs, RDF Data Model, variety of RDF Serialization Formats[1][2][3], and Shared Data Dictionaries / Schemas / Ontologies [1][2][3][4][5]) it's now possible to Virtualize enterprise data from the Physical Storage Level, through the Logical Data Management Levels (Relational), up to a Concrete Conceptual Model (Graph) without operating system, development environment or framework, or database engine lock-in.

Next Steps

We produce a shared ontology for the CRM and Business Reporting Domains. I hope this experiment clarifies how this is quite achievable by converting XML Schemas to RDF Data Dictionaries (RDF Schemas or Ontologies). Stay tuned :-)

Also watch TimBL amplify and articulate Linked Data value in a recent interview.

Other Related Matters

To deliver a mechanism that facilitates the crystallization of this reality is a contribution of boundless magnitude (as we shall all see in due course). Thus, it is easy to understand why even "her majesty", the queen of England, simply had to get in on the act and appoint TimBL to the "British Order of Merit" :-)

Note: All of the demos above now work with IE & Safari (a "remember what Virtuoso is epiphany") by simply putting Virtuoso's DBMS hosted XSLT engine to use :-) This also applies to my earlier collection of demos from the Hello Data Web and other Data Web & Linked Data related demo style posts.

Now that broader understanding of the Semantic Data Web is emerging, I would like to revisit the issue of "Data Spaces".

A Data Space is a place where Data Resides. It isn't inherently bound to a specific Data Model (Concept Oriented, Relational, Hierarchical etc..). Neither is it implicitly an access point to Data, Information, or Knowledge (the perception is purely determined through the experiences of the user agents interacting with the Data Space.

A Web Data Space is a Web accessible Data Space.

Real world example:

Today we increasing perform one of more of the following tasks as part of our professional and personal interactions on the Web:

1. Blog via many service providers or personally managed weblog platforms
2. Create Event Calendars via Upcoming.com and Eventful
3. Maintain and participate in Social Networks (e.g. Facebook, Orkut, MySpace)
4. Create and Participate in Discussions (note: when you comment on blogs or wikis for instance, you are participating in, or creating, a conversation)
5. Track news by subscribing to RSS 1.0, RSS 2.0, or Atom Feeds
6. Share Bookmarks & Tags via Del.icio.us and other Services
7. Share Photos via Flickr
8. Buy, Review, or Search for books via Amazon
9. Participates in auctions via eBay
10. Search for data via Google (of course!)

John Breslin has nice a animation depicting the creation of Web Data Spaces that drives home the point.

Unfortunately, what isn't as obvious to many netizens, is the fact that each of the activities above results in the creation of data that is put into some context by you the user. Even worse, you eventually realize that the service providers aren't particularly willing, or capable of, giving you unfettered access to your own data. Of course, this isn't always by design as the infrastructure behind the service can make this a nightmare from security and/or load balancing perspectives. Irrespective of cause, we end up creating our own "Data Spaces" all over the Web without a coherent mechanism for accessing and meshing these "Data Spaces".

Data Spaces on the Web that provide granular access to RDF Data.

Short History

In anticipation of this the "Web Data Silo" challenge (an issue that we tackled within internal enterprise networks for years) we commenced the development (circa. 2001) of a distributed collaborative application suite called OpenLink Data Spaces (ODS). The project was never released to the public since the problems associated with the deliberate or inadvertent creation of Web Data silos hadn't really materialized (silos only emerged in concreted form after the emergence of the Blogosphere and Web 2.0). In addition, there wasn't a clear standard Query Language for the RDF based Web Data Model (i.e. the SPARQL Query Language didn't exist).

Today, ODS is delivered as a packaged solution (in Open Source and Commercial flavors) that alleviates the pain associated with Data Space Silos that exist on the Web and/or behind corporate firewalls. In either scenario, ODS simply allows you to create Open and Secure Data Spaces (via it's suite of applications) that expose data via SQL, RDF, XML oriented data access and data management technologies. Of course it also enables you to integrates transparently with existing 3rd party data space generators (Blogs, Wikis, Shared Bookmrks, Discussion etc. services) by supporting industry standards that cover:

1. Content Publishing - Atom, Moveable Type, MetaWeblog, Blogger protocols
2. Content Syndication Formats - RSS 1.0, RSS 2.0, Atom, OPML etc.
3. Data Management - SQL, RDF, XML, Free Text
4. Data Access - SQL, SPARQL, GData, Web Services (SOAP or REST styles), WebDAV/HTTP
5. Semantic Data Web Middleware - GRDDL, XSLT, SPARQL, XPath/XQuery, HTTP (Content Negotiation) for producing RDF from non RDF Data ((X)HTML, Microformats, XML, Web Services Response Data etc).

Thus, by installing ODS on your Desktop, Workgroup, Enterprise, or public Web Server, you end up with a very powerful solution for creating Open Data access oriented presence on the "Semantic Data Web" without incurring any of the typically assumed "RDF Tax".

Naturally, ODS is built atop Virtuoso and of course it exploits Virtuoso's feature-set to the max. It's also beginning to exploit functionality offered by the OpenLink Ajax Toolkit (OAT).

Some Definitions (as per usual):

RDF Middleware (as defined in this context) is about producing RDF from non RDF Data Sources. This implies that you can use non RDF Data Sources (e.g. (X)HTML Web Pages, (X)HTML Web Pages hosting Microformats, and even Web Services such as those from Google, Del.icio.us, Flickr etc..) as Semantic Web Data Source URIs (pointers to RDF Data).

In this post I would like to provide a similar perspective on this ability to treat non RDF as RDF from RDF Browser perspective.

First off, what's an RDF Browser?

An RDF Browser is a piece of technology that enables you to Browse RDF Data Sources by way of Data Link Traversal. The key difference between this approach and traditional browsing is that Data Links are typed (they possess inherent meaning and context) whereas traditional links are untyped (although universally we have been trained to type them as links to Blurb in the form of (X)HTML pages or what is popularly called "Web Content".).

There are a number of RDF Browsers that I am aware off (note: pop me a message directly of by way of a comment to this post if you have a browser that I am unaware of), and they include (in order of creation and availability):

1. Tabulator
2. DISCO - Hyperdata Browser
3. OpenLink Ajax Toolkit's RDF Browser (a component of the OAT Javascript Toolkit)

Each of the browsers above can consume the services of Triplr or the Virtuoso Sponger en route to unveiling a RDF Data that is traversable via URI dereferencing (HTTP GETing the data exposed by the Data Pointer). Thus you can cut&paste the following into each of the aforementioned RDF Browsers:

1. Triplr's RDF Data (Triples) extractions from Dan Connolly's Home Page
2. The Virtuoso Sponger's RDF Data (Triples) extractions from Dan Connolly's Home Page

Since we are all time challenged (naturally!) you can also just click on these permalinks for the OAT RDF Browser demos:

1. Permalink for Triplr's RDF Data (Triples) extractions from Dan Connolly's Home Page
2. Permalink for the Virtuoso Sponger's RDF Data (Triples) extractions from Dan Connolly's Home Page

Quick Definitions:

Mashups - Brute force joining of disparate Web Data

Meshups - Natural joining of disparate Web Data

Reasons for the distinction:

Mashups are Data Model oblivious.

Meshups are Data Model driven.

Examples:

Mashups are based on RSS 2.0 most of the time (RSS 2.0 is at best a Tree Structure that contains untyped or meaning challenged links.

Meshups are RDF based and the data is self describing since the links are typed (posses inherent meaning thereby providing context).

So what? You may be thinking.

For starters, I can quite easily Mesh data from Googlebase (which emits RSS 2.0 or Atom) and other data sources with the Mapping Services from Yahoo!

I can achieve this in minutes without writing a single line of code. I can do it because of the Data Model prowess of RDF (self-describing instance-data), the data interchange and transformation power of XML and XSLT respectively, the inherent power of XML based Web Services (REST or SOAP), and of course, having a Hybrid Server product like Virtuoso at my disposal that delivers a cross platform solution for exploiting all of these standards coherently.

I can share the self-describing describing data source that serves my Meshup. Try reusing the data presented by a Mashup via the same URL that you used to locate Mashup to get my drift.

Demo Links:

1. Googlebase Query URL as an RDF Data Source
2. Perform a simple Data Mesh by adding (via link copy and paste) this Upcoming.org Query Services URL for Ajax Events to the RDF Browsers list of Data Sources (paste into the Data Source URI input field).

What does this all mean?

"Context" is the catalyst of the burgeoning Data Web (Semantic Web Layer - 1). It's the emerging appreciation of "Context" that is driving the growing desire to increment Web versions from 2.0 to 3.0. It also the the very same "Context" that has been a preoccupation of Semantic Web vision since its inception.

The journey towards a more Semantic Web is all inclusive (all "ANDs" and no "ORs" re. participation).

The Semantic Web is self-annotating. Web 2.0 has provided a huge contribution to the self annotation effort: on the Web we now have Data Spaces for Bookmarks (e.g del.icio.us), Image Galleries ( e.g Flickr), Discussion Forums (remember those comments associated with blog posts? ditto the pingbacks and trackbacks?), People Profiles (FOAF, XFN, del.icio.us, and those crumbling walled-gardens around many Social Networks), and more..

A Web without granular access to Data is simply not a Web worth having (think about the menace of click-fraud and spam).

What Is Virtuoso?

A powerful next generation server product that implements otherwise distinct server functionality within a single server product. Think of Virtuoso as the server software analog of a dual core processor where each core represents a traditional server functionality realm.

Where did it come from?

The Virtuoso History page tells the whole story.

What Functionality Does It Provide?

1. Object-Relational DBMS Engine (ORDBMS like PostgreSQL and DBMS engine like MySQL)

2. XML Data Management (with support for XQuery, XPath, XSLT, and XML Schema)

3. RDF Triple Store (or Database) that supports SPARQL (Query Language, Transport Protocol, and XML Results Serialization format)

4. Service Oriented Architecture (it combines a BPEL Engine with an ESB)

5. Web Application Server (supports HTTP/WebDAV)

6. NNTP compliant Discussion Server

90% of the aforementioned functionality has been available in Virtuoso since 2000 with the RDF Triple Store being the only 2006 item.

What Platforms are Supported

The Virtuoso build scripts have been successfully tested on Mac OS X (Universal Binary Target), Linux, FreeBSD, and Solaris (AIX, HP-UX, and True64 UNIX will follow soon). A Windows Visual Studio project file is also in the works (ETA some time this week).

Why Open Source?

Simple, there is no value in a product of this magnitude remaining the "best kept secret". That status works well for our competitors, but absolutely works against the legions of new generation developers, systems integrators, and knowledge workers that need to be aware of what is actually achievable today with the right server architecture.

What Open Source License is it under?

GPL version 2.

What's the business model?

Dual licensing.

The Open Source version of Virtuoso includes all of the functionality listed above. While the Virtual Database (distributed heterogeneous join engine) and Replication Engine (across heterogeneous data sources) functionality will only be available in the commercial version.

Where is the Project Hosted?

On SourceForge.

Is there a product Blog?

Of course!

Up until this point, the Virtuoso Product Blog has been a covert live demonstration of some aspects of Virtuoso (Content Management). My Personal Blog and the Virtuoso Product Blog are actual Virtuoso instances, and have been so since I started blogging in 2003.

Is There a product Wiki?

Sure! The Virtuoso Product Wiki is also an instance of Virtuoso demonstrating another aspect of the Content Management prowess of Virtuoso.

What About Online Documentation?

Yep! Virtuoso Online Documentation is hosted via yet another Virtuoso instance. This particular instance also attempts to demonstrate Free Text search combined with the ability to repurpose well formed content in a myriad of forms (Atom, RSS, RDF, OPML, and OCS).

What about Tutorials and Demos?

The Virtuoso Online Tutorial Site has operated as a live demonstration and tutorial portal for a numbers of years. During the same timeframe (circa. 2001) we also assembled a few Screencast style demos (their look feel certainly show their age; updates are in the works).

BTW - We have also updated the Virtuoso FAQ and also released a number of missing Virtuoso White Papers (amongst many long overdue action items).

Techniques for presenting and managing syndication XML (feeds) are disclosed. In one embodiment, a user can modify how a feed is displayed, such as which content (and how much) is displayed, in what order, and how it is formatted. In another embodiment, a modification regarding how a feed is displayed is stored so that it can be used again at a later time. In yet another embodiment, a user can create a custom feed through aggregation and/or filtering of existing feeds. Aggregation includes, for example, merging the articles of multiple feeds to form a new feed. Filtering includes, for example, selecting a subset of articles of a feed based on whether they satisfy a search query. In yet another embodiment, a user can find articles by entering a search query into a search engine that searches feeds, which will identify one or more articles that satisfy the query.

Clearly Apple don't seem to understand the world of XML, so let me give them a quick recap:

1. XML enables separation of Data and Formating

2. It facilitates Data Representation, Transformation (XSLT), Exchange (syndication and subscription), and Modeling (languages, protocols, data models, amongst other things)

3. It is inherently open

4. You can't patent its essence through the back door!

The Blogosphere is a Galaxy within Cyberspace comprised of Solar systems of Blogs that revolve around X-list bloggers, Topics, or more recently Tags; through the gravitational pull of links to RSS (today), Atom (in due course), and RDF (the future).

Unfortunately, Apple (a major late-comer to RSS) doesn't seem to understand that "RSS content search, aggregation and transformation" is practically the same thing as "XML search, aggregation and transformation". Subject matter covered extensively by XML based languages such as XSLT, XPath, XPointer, and XQuery.

Without XML there would be no RSS (as we know it today), and without RSS there would be no Blogosphere.

Repurposing Blogosphere content isn't a novel invention at all. Therefore, filing a patent along such lines is simply uncool by Apple's standards (like the inextricable binding of iWeb to .mac that was touted as innovative and open).

Final note: this blog is driven by a database engine that has understood XML for a long time. This blog has been my live demo of this fact since its inception. Here are a few things that it has done for a very long time (talking prior art here):

- Repurpose content on the fly from SQL and XML data sources to produce all the syndication and subscription gems you see on the Blog Home Page

- Offer a search feature that enables visitors to query blog archives using Free Text, XQuery, XPath (all transformation technologies alongside XSLT).

- Produce Query Results that are "Open" for reuse outside the domain of this blog using standard syndication formats.

Piggy Bank 2.0 Beta

New release of Piggy Bank, the Semantic Web extension for Firefox. It harvests data as you browse (when you click a status bar indicator), which can later be searched and viewed in a facetted browser.

The docs have come along some too -

Piggy Bank can collect pure information in the following cases:

1. The web page has invisible link(s) to RDF data (encoded in RDF/XML or N3 formats).
2. The web page exports an RSS feeds.
3. The address of the web page is a file:/ URL pointing to a directory.
4. Piggy Bank has a "screen scraper"€ [XSLT or Javascript] that can re-structure the web page HTML code into RDF data.

There's a tutorial on writing Javascript screenscrapers on the site, nice touch.

I have also added an architecture diagram to accelerate comprehension (a picture speaks a thousand words...):

Here goes:

Blog Editing

I can use any editor that supports the following Blog Post APIs:

- Moveable Type

- Meta Weblog

- Blogger

Typically I use Virtuoso (which has an unreleased WYSIWYG blog post editor), Newzcrawler, ecto, Zempt, or w.bloggar for my posts. If a post is of interest to me, or relevant to our company or customers I tend to perform one of the following tasks:

- Generate a post using the "Blog This" feature of my blog editor

- Write a new post that was triggered by a previously read post etc.

Either way, the posts end up in our company wide blog server that is Virtuoso based (more about this below). The internal blog server automatically categorizes my blog posts, and automagically determines which posts to upstream to other public blogs that I author (e.g http://kidehen.typepad.com ) or co-author (e.g http://www.openlinksw.com/weblogs/uda and http://www.openlinksw.com/weblogs/virtuoso ). I write once and my posts are dispatched conditionally to multiple outlets.

RSS/Atom/RDF Aggregation & Reading

I discover, subscribe to, and view blog feeds using Newzcrawler (primarily), and from time to time for experimentation and evaluation purposes I use RSS Bandit, FeedDemon, and Bloglines. I am in the process of moving this activity over to Virtuoso completely due to the large number of feeds that I consume on a daily basis (scalability is a bit of a problem with current aggregators).

Blog Publishing

When you visit my blog you are experiencing the  soon to be released Virtuoso Blog Publishing engine first hand, which is how WebDAV, SQLX, XQuery/XPath, and Free Text etc. come into the mix.

Each time I create a post internally, or subscribe to an external feed, the data ends up in Virtuoso's SQL Engine (this is how we handle some of the obvious scalability challenges associated with large subscription counts). This engine is SQL2000N based, which implies that it can transform SQL to XML on the fly using recent extensions to SQL in the form of SQLX (prior to the emergence of this standard we used the FOR XML SQL syntax extensions for the same result). It also has its own in-built XSLT processor (DB Engine resident), and validating XML parser (with support for XML Schema).  Thus, my RSS/RDF/Atom archives, FOAF, BlogRoll, OPML, and OCS blog syndication gems are all live examples of SQLX documents that leverage Virtuoso's WebDAV engine for exposure to Blog Clients.

Blog Search

When you search for blog posts using the basic or advanced search features of my blog, you end up interacting with one of the following methods of querying data hosted in Virtuoso: Free Text Search, XPath, or XQuery. The result sets produced by the search feature uses SQLX to produce subscription gems (RSS/Atom/RDF/OpenSearch) and URIs that enable dynamic tracking of my posts using your search keywords.

BTW - the http://www.openlinksw.com/blog/~kidehen blog home page exists as a result of Virtuoso's Virtual Domain / Multi-Homing Web Server functionality. The entire site resides in an Object Relational DBMS, and I can take my DB file across Windows, Solaris, Linux, Mac OS X, FreeBSD, AIX, HP-UX, IRIX, and SCO UnixWare without missing a single beat! All I have to do is instantiate my Virtuoso server and my weblog is live.

By Peter Sefton, XML.org

The author explores some of the ways that OpenOffice.org's Writer application is open to customization and configuration. He coveres a few techniques that will be of interest to template maintainers working with OpenOffice.org writer: how to crack open the file format, how to maintain large sets of styles, and how to customize menus and macros, all without using anything except standard tools, zip, an XSLT processor, and a text editor. All this can, of course, be further automated with a programming language of some kind, even a batch file.

There are some changes coming in version 2 of OpenOffice.org, but all these techniques will be forwards compatible, although some things like the location and name of the menu-bar files look like they will change. If you are also trying to store and manipulate content in XML but want to use a word processing environment for authoring, then well-crafted templates are even more important.

http://www.xml.com/pub/a/2005/01/26/hacking-ooo.html

See also the OpenDocument 1.0 CD: http://xml.coverpages.org/ni2005-01-04-a.html

This also includes Weblog Editing and Posting I believe.

Full Spec

A few days ago, I was waiting at a dentist and trying to kill the time thought of using my Nokia 3650 (with Doris HTML browser) to have a look at your site, to see if there's anything there that might put my mind on other matters than the precarious closeness of the dentist drilling machines! And what I found out was how hard it was to get to the actual content on your site...I had to scroll through all of what is usually hidden...after about 10 minutes or so I finally got to the content. Any way to redesign it, so that content gets loaded first or putting up a mobile lightweight version?

Excellent point. I thought about this for five seconds and realized that Rob could solve this problem for himself -- and for others -- in a very simple way. I pointed him at the solution, and he picked up the ball and ran with it. ...

My blog is currently available in two XML flavors: the standard feed and the extended feed. My suggestion to Rob was to write an XSLT transform for one or the other, and pipe the XML content through it (using the W3C's public XSLT transformation service) to create a lightweight HTML rendering.

Here is the XSLT file Rob wrote. Here's how it renders my standard feed. Here's how it renders my extended feed.

As Rob notes in his writeup, there was a problem with the extended feed, so originally he was only able to pipe the standard feed to his Nokia. But that was my fault, not his. I kicked my setup and it seems to be working properly now. Rob's conclusion:

Wow, that was pretty simple and quite powerful. The power of this kind of ad hoc scripting never ceases to amaze me! [Clarity's Blog]

[via Jon's Radio]

Very cool! The power of XSLT! Now I am sure we can see how Virtuoso would extend this further? In short I will try to have this become an attribute of my Blog.

Virtuoso could enable this site to automatically determine what type of User Agent (clients such as Web Browsers)

A few days ago, I was waiting at a dentist and trying to kill the time thought of using my Nokia 3650 (with Doris HTML browser) to have a look at your site, to see if there's anything there that might put my mind on other matters than the precarious closeness of the dentist drilling machines! And what I found out was how hard it was to get to the actual content on your site...I had to scroll through all of what is usually hidden...after about 10 minutes or so I finally got to the content. Any way to redesign it, so that content gets loaded first or putting up a mobile lightweight version?

Excellent point. I thought about this for five seconds and realized that Rob could solve this problem for himself -- and for others -- in a very simple way. I pointed him at the solution, and he picked up the ball and ran with it. ...

My blog is currently available in two XML flavors: the standard feed and the extended feed. My suggestion to Rob was to write an XSLT transform for one or the other, and pipe the XML content through it (using the W3C's public XSLT transformation service) to create a lightweight HTML rendering.

Here is the XSLT file Rob wrote. Here's how it renders my standard feed. Here's how it renders my extended feed.

As Rob notes in his writeup, there was a problem with the extended feed, so originally he was only able to pipe the standard feed to his Nokia. But that was my fault, not his. I kicked my setup and it seems to be working properly now. Rob's conclusion:

Wow, that was pretty simple and quite powerful. The power of this kind of ad hoc scripting never ceases to amaze me! [Clarity's Blog]

[via Jon's Radio]

Very cool! The power of XSLT! Now I am sure we can see how Virtuoso would extend this further? In short I will try to have this become an attribute of my Blog.

Virtuoso could enable this site to automatically determine what type of User Agent (clients such as Web Browsers) is being used by the visitor and then automatically associated the required XSTL stylesheet for the User Agent.

This was one of the very basic Virtuoso XML and XSLT demos (circa 2000-2001).

Blogging is going to provide a very fluid demo canvas for Virtuoso as this article demonstrates.

XML and relational databases are both technologies for structuring, cataloguing and processing data. If data has a regular and atomic structure, it is more appropriate and efficient to use a database than XML.

Databases store data, XML is not a storage mechanism, it is a storage format (amongst its many capabilities).

In this case, why would you wish to go to the trouble of converting such data from a database into XML and vice versa? Reasons include:

• XML is easy to convert further into different formats as required: e.g HTML, PDF, and plain text. This gives a flexibility to web applications where data can be searched for and accessed from the database, and then formatted for output in different formats using e.g XSL.

XML seperates data from formatting (and programming logic). XSL is now broken down into two parts; XSLT