A simple guide usable by any Perl developer seeking to exploit SPARQL without hassles.

Why?

SPARQL is a powerful query language, results serialization format, and an HTTP based data access protocol from the W3C. It provides a mechanism for accessing and integrating data across Deductive Database Systems (colloquially referred to as triple or quad stores in Semantic Web and Linked Data circles) -- database systems (or data spaces) that manage proposition oriented records in 3-tuple (triples) or 4-tuple (quads) form.

How?

SPARQL queries are actually HTTP payloads (typically). Thus, using a RESTful client-server interaction pattern, you can dispatch calls to a SPARQL compliant data server and receive a payload for local processing.

Steps:

1. Determine which SPARQL endpoint you want to access e.g. DBpedia or a local Virtuoso instance (typically: http://localhost:8890/sparql).
2. If using Virtuoso, and you want to populate its quad store using SPARQL, assign "SPARQL_SPONGE" privileges to user "SPARQL" (this is basic control, more sophisticated WebID based ACLs are available for controlling SPARQL access).

Script:

#
# Demonstrating use of a single query to populate a 
# Virtuoso Quad Store via Perl. 
#

# 
# HTTP URL is constructed accordingly with CSV query results format as the default via mime type.
#

use CGI qw/:standard/;
use LWP::UserAgent;
use Data::Dumper;
use Text::CSV_XS;

sub sparqlQuery(@args) {
  my $query=shift;
  my $baseURL=shift;
  my $format=shift;
	
	%params=(
		"default-graph" => "", "should-sponge" => "soft", "query" => $query,
		"debug" => "on", "timeout" => "", "format" => $format,
		"save" => "display", "fname" => ""
	);
	
	@fragments=();
	foreach $k (keys %params) {
		$fragment="$k=".CGI::escape($params{$k});
		push(@fragments,$fragment);
	}
	$query=join("&", @fragments);
	
	$sparqlURL="${baseURL}?$query";
	
	my $ua = LWP::UserAgent->new;
	$ua->agent("MyApp/0.1 ");
	my $req = HTTP::Request->new(GET => $sparqlURL);
	my $res = $ua->request($req);
	$str=$res->content;
	
	$csv = Text::CSV_XS->new();
	
	foreach $line ( split(/^/, $str) ) {
		$csv->parse($line);
		@bits=$csv->fields();
	  push(@rows, [ @bits ] );
	}
	return \@rows;
}


# Setting Data Source Name (DSN)

$dsn="http://dbpedia.org/resource/DBpedia";

# Virtuoso pragmas for instructing SPARQL engine to perform an HTTP GET using the IRI in
# FROM clause as Data Source URL en route to DBMS
# record Inserts.

$query="DEFINE get:soft \"replace\"\n

# Generic (non Virtuoso specific SPARQL
# Note: this will not add records to the 
# DBMS 

SELECT DISTINCT * FROM <$dsn> WHERE {?s ?p ?o}"; 

$data=sparqlQuery($query, "http://localhost:8890/sparql/", "text/csv");

print "Retrieved data:\n";
print Dumper($data);

Output

Retrieved data:
$VAR1 = [
          [
            's',
            'p',
            'o'
          ],
          [
            'http://dbpedia.org/resource/DBpedia',
            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',
            'http://www.w3.org/2002/07/owl#Thing'
          ],
          [
            'http://dbpedia.org/resource/DBpedia',
            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',
            'http://dbpedia.org/ontology/Work'
          ],
          [
            'http://dbpedia.org/resource/DBpedia',
            'http://www.w3.org/1999/02/22-rdf-syntax-ns#type',
            'http://dbpedia.org/class/yago/Software106566077'
          ],
...

Conclusion

CSV was chosen over XML (re. output format) since this is about a "no-brainer installation and utilization" guide for a Perl developer that already knows how to use Perl for HTTP based data access within HTML. SPARQL just provides an added bonus to URL dexterity (delivered via URI abstraction) with regards to constructing Data Source Names or Addresses.

Related

• RDF::Query::Client Guide
• SPARQL Guide for the Perl Developer
• SPARQL Guide for the PHP Developer
• SPARQL Guide for the Python Developer
• SPARQL Guide for the Ruby Developer
• Simple Guide for using SPARQL with Virtuoso
• General SPARQL Tutorial Collection
• Virtuoso Specific SPARQL Tutorial Collection
• The URI, URL, and Linked Data Meme's Generic HTTP URI.

A simple guide usable by any Javascript developer seeking to exploit SPARQL without hassles.

Why?

SPARQL is a powerful query language, results serialization format, and an HTTP based data access protocol from the W3C. It provides a mechanism for accessing and integrating data across Deductive Database Systems (colloquially referred to as triple or quad stores in Semantic Web and Linked Data circles) -- database systems (or data spaces) that manage proposition oriented records in 3-tuple (triples) or 4-tuple (quads) form.

How?

SPARQL queries are actually HTTP payloads (typically). Thus, using a RESTful client-server interaction pattern, you can dispatch calls to a SPARQL compliant data server and receive a payload for local processing.

Steps:

1. Determine which SPARQL endpoint you want to access e.g. DBpedia or a local Virtuoso instance (typically: http://localhost:8890/sparql).
2. If using Virtuoso, and you want to populate its quad store using SPARQL, assign "SPARQL_SPONGE" privileges to user "SPARQL" (this is basic control, more sophisticated WebID based ACLs are available for controlling SPARQL access).

Script:

/*
Demonstrating use of a single query to populate a # Virtuoso Quad Store via Javascript. 
*/

/* 
HTTP URL is constructed accordingly with JSON query results format as the default via mime type.
*/

function sparqlQuery(query, baseURL, format) {
	if(!format)
		format="application/json";
	var params={
		"default-graph": "", "should-sponge": "soft", "query": query,
		"debug": "on", "timeout": "", "format": format,
		"save": "display", "fname": ""
	};
	
	var querypart="";
	for(var k in params) {
		querypart+=k+"="+encodeURIComponent(params[k])+"&";
	}
	var queryURL=baseURL + '?' + querypart;
	if (window.XMLHttpRequest) {
  	xmlhttp=new XMLHttpRequest();
  }
  else {
  	xmlhttp=new ActiveXObject("Microsoft.XMLHTTP");
  }
  xmlhttp.open("GET",queryURL,false);
  xmlhttp.send();
  return JSON.parse(xmlhttp.responseText);
}

/*
setting Data Source Name (DSN)
*/

var dsn="http://dbpedia.org/resource/DBpedia";

/*
Virtuoso pragma "DEFINE get:soft "replace" instructs Virtuoso SPARQL engine to perform an HTTP GET using the IRI in FROM clause as Data Source URL with regards to 
DBMS record inserts
*/

var query="DEFINE get:soft \"replace\"\nSELECT DISTINCT * FROM <"+dsn+"> WHERE {?s ?p ?o}"; 
var data=sparqlQuery(query, "/sparql/");

Output

Place the snippet above into the <script/> section of an HTML document to see the query result.

Conclusion

JSON was chosen over XML (re. output format) since this is about a "no-brainer installation and utilization" guide for a Javascript developer that already knows how to use Javascript for HTTP based data access within HTML. SPARQL just provides an added bonus to URL dexterity (delivered via URI abstraction) with regards to constructing Data Source Names or Addresses.

Related

• SPARQL Guide for the PHP Developer
• SPARQL Guide for the Python Developer
• SPARQL Guide for the Ruby Developer
• Simple Guide for using SPARQL with Virtuoso
• General SPARQL Tutorial Collection
• Virtuoso Specific SPARQL Tutorial Collection
• The URI, URL, and Linked Data Meme's Generic HTTP URI.

A simple guide usable by any PHP developer seeking to exploit SPARQL without hassles.

Why?

SPARQL is a powerful query language, results serialization format, and an HTTP based data access protocol from the W3C. It provides a mechanism for accessing and integrating data across Deductive Database Systems (colloquially referred to as triple or quad stores in Semantic Web and Linked Data circles) -- database systems (or data spaces) that manage proposition oriented records in 3-tuple (triples) or 4-tuple (quads) form.

How?

SPARQL queries are actually HTTP payloads (typically). Thus, using a RESTful client-server interaction pattern, you can dispatch calls to a SPARQL compliant data server and receive a payload for local processing e.g. local object binding re. PHP.

Steps:

1. From your command line execute: aptitude search '^PHP26', to verify PHP is in place
2. Determine which SPARQL endpoint you want to access e.g. DBpedia or a local Virtuoso instance (typically: http://localhost:8890/sparql).
3. If using Virtuoso, and you want to populate its quad store using SPARQL, assign "SPARQL_SPONGE" privileges to user "SPARQL" (this is basic control, more sophisticated WebID based ACLs are available for controlling SPARQL access).

Script:

#!/usr/bin/env php
<?php
#
# Demonstrating use of a single query to populate a # Virtuoso Quad Store via PHP. 
#

# HTTP URL is constructed accordingly with JSON query results format in mind.

function sparqlQuery($query, $baseURL, $format="application/json")

  {
	$params=array(
		"default-graph" =>  "",
		"should-sponge" =>  "soft",
		"query" =>  $query,
		"debug" =>  "on",
		"timeout" =>  "",
		"format" =>  $format,
		"save" =>  "display",
		"fname" =>  ""
	);

	$querypart="?";	
	foreach($params as $name => $value) 
  {
		$querypart=$querypart . $name . '=' . urlencode($value) . "&";
	}
	
	$sparqlURL=$baseURL . $querypart;
	
	return json_decode(file_get_contents($sparqlURL));
};



# Setting Data Source Name (DSN)
$dsn="http://dbpedia.org/resource/DBpedia";

#Virtuoso pragmas for instructing SPARQL engine to perform an HTTP GET
#using the IRI in FROM clause as Data Source URL

$query="DEFINE get:soft \"replace\"
SELECT DISTINCT * FROM <$dsn> WHERE {?s ?p ?o}"; 

$data=sparqlQuery($query, "http://localhost:8890/sparql/");

print "Retrieved data:\n" . json_encode($data);

?>

Output

Retrieved data:
  {"head":
  {"link":[],"vars":["s","p","o"]},
  "results":
		{"distinct":false,"ordered":true,
		"bindings":[
			{"s":
			{"type":"uri","value":"http:\/\/dbpedia.org\/resource\/DBpedia"},"p":
			{"type":"uri","value":"http:\/\/www.w3.org\/1999\/02\/22-rdf-syntax-ns#type"},"o":
			{"type":"uri","value":"http:\/\/www.w3.org\/2002\/07\/owl#Thing"}},
			{"s":
			{"type":"uri","value":"http:\/\/dbpedia.org\/resource\/DBpedia"},"p":
			{"type":"uri","value":"http:\/\/www.w3.org\/1999\/02\/22-rdf-syntax-ns#type"},"o":
			{"type":"uri","value":"http:\/\/dbpedia.org\/ontology\/Work"}},
			{"s":
			{"type":"uri","value":"http:\/\/dbpedia.org\/resource\/DBpedia"},"p":
			{"type":"uri","value":"http:\/\/www.w3.org\/1999\/02\/22-rdf-syntax-ns#type"},"o":
			{"type":"uri","value":"http:\/\/dbpedia.org\/class\/yago\/Software106566077"}},
...

Conclusion

JSON was chosen over XML (re. output format) since this is about a "no-brainer installation and utilization" guide for a PHP developer that already knows how to use PHP for HTTP based data access. SPARQL just provides an added bonus to URL dexterity (delivered via URI abstraction) with regards to constructing Data Source Names or Addresses.

Related

• SPARQL Guide for the Python Developer
• SPARQL Guide for the Ruby Developer
• Simple Guide for using SPARQL with Virtuoso
• General SPARQL Tutorial Collection
• Virtuoso Specific SPARQL Tutorial Collection
• The URI, URL, and Linked Data Meme's Generic HTTP URI.

A simple guide usable by any Python developer seeking to exploit SPARQL without hassles.

Why?

SPARQL is a powerful query language, results serialization format, and an HTTP based data access protocol from the W3C. It provides a mechanism for accessing and integrating data across Deductive Database Systems (colloquially referred to as triple or quad stores in Semantic Web and Linked Data circles) -- database systems (or data spaces) that manage proposition oriented records in 3-tuple (triples) or 4-tuple (quads) form.

How?

SPARQL queries are actually HTTP payloads (typically). Thus, using a RESTful client-server interaction pattern, you can dispatch calls to a SPARQL compliant data server and receive a payload for local processing e.g. local object binding re. Python.

Steps:

1. From your command line execute: aptitude search '^python26', to verify Python is in place
2. Determine which SPARQL endpoint you want to access e.g. DBpedia or a local Virtuoso instance (typically: http://localhost:8890/sparql).
3. If using Virtuoso, and you want to populate its quad store using SPARQL, assign "SPARQL_SPONGE" privileges to user "SPARQL" (this is basic control, more sophisticated WebID based ACLs are available for controlling SPARQL access).

Script:

#!/usr/bin/env python
#
# Demonstrating use of a single query to populate a # Virtuoso Quad Store via Python. 
#

import urllib, json

# HTTP URL is constructed accordingly with JSON query results format in mind.

def sparqlQuery(query, baseURL, format="application/json"):
	params={
		"default-graph": "",
		"should-sponge": "soft",
		"query": query,
		"debug": "on",
		"timeout": "",
		"format": format,
		"save": "display",
		"fname": ""
	}
	querypart=urllib.urlencode(params)
	response = urllib.urlopen(baseURL,querypart).read()
	return json.loads(response)

# Setting Data Source Name (DSN)
dsn="http://dbpedia.org/resource/DBpedia"

# Virtuoso pragmas for instructing SPARQL engine to perform an HTTP GET
# using the IRI in FROM clause as Data Source URL

query="""DEFINE get:soft "replace"
SELECT DISTINCT * FROM <%s> WHERE {?s ?p ?o}""" % dsn 

data=sparqlQuery(query, "http://localhost:8890/sparql/")

print "Retrieved data:\n" + json.dumps(data, sort_keys=True, indent=4)

#
# End

Output

Retrieved data:
{
    "head": {
        "link": [], 
        "vars": [
            "s", 
            "p", 
            "o"
        ]
    }, 
    "results": {
        "bindings": [
            {
                "o": {
                    "type": "uri", 
                    "value": "http://www.w3.org/2002/07/owl#Thing"
                }, 
                "p": {
                    "type": "uri", 
                    "value": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type"
                }, 
                "s": {
                    "type": "uri", 
                    "value": "http://dbpedia.org/resource/DBpedia"
                }
            }, 
...

Conclusion

JSON was chosen over XML (re. output format) since this is about a "no-brainer installation and utilization" guide for a Python developer that already knows how to use Python for HTTP based data access. SPARQL just provides an added bonus to URL dexterity (delivered via URI abstraction) with regards to constructing Data Source Names or Addresses.

Related

• SPARQL Guide for the Ruby Developer
• Simple Guide for using SPARQL with Virtuoso
• General SPARQL Tutorial Collection
• Virtuoso Specific SPARQL Tutorial Collection
• The URI, URL, and Linked Data Meme's Generic HTTP URI.

A simple guide usable by any Ruby developer seeking to exploit SPARQL without hassles.

Why?

SPARQL is a powerful query language, results serialization format, and an HTTP based data access protocol from the W3C. It provides a mechanism for accessing and integrating data across Deductive Database Systems (colloquially referred to as triple or quad stores in Semantic Web and Linked Data circles) -- database systems (or data spaces) that manage proposition oriented records in 3-tuple (triples) or 4-tuple (quads) form.

How?

SPARQL queries are actually HTTP payloads (typically). Thus, using a RESTful client-server interaction pattern, you can dispatch calls to a SPARQL compliant data server and receive a payload for local processing e.g. local object binding re. Ruby.

Steps:

1. From your command line execute: aptitude search '^ruby', to verify Ruby is in place
2. Determine which SPARQL endpoint you want to access e.g. DBpedia or a local Virtuoso instance (typically: http://localhost:8890/sparql).
3. If using Virtuoso, and you want to populate its quad store using SPARQL, assign "SPARQL_SPONGE" privileges to user "SPARQL" (this is basic control, more sophisticated WebID based ACLs are available for controlling SPARQL access).

Script:

#!/usr/bin/env ruby
#
# Demonstrating use of a single query to populate a # Virtuoso Quad Store. 
#

require 'net/http'
require 'cgi'
require 'csv'

#
# We opt for CSV based output since handling this format is straightforward in Ruby, by default.
# HTTP URL is constructed accordingly with CSV as query results format in mind.

def sparqlQuery(query, baseURL, format="text/csv")
	params={
		"default-graph" => "",
		"should-sponge" => "soft",
		"query" => query,
		"debug" => "on",
		"timeout" => "",
		"format" => format,
		"save" => "display",
		"fname" => ""
	}
	querypart=""
	params.each { |k,v|
		querypart+="#{k}=#{CGI.escape(v)}&"
	}
  
	sparqlURL=baseURL+"?#{querypart}"
	
	response = Net::HTTP.get_response(URI.parse(sparqlURL))

	return CSV::parse(response.body)
	
end

# Setting Data Source Name (DSN)

dsn="http://dbpedia.org/resource/DBpedia"

#Virtuoso pragmas for instructing SPARQL engine to perform an HTTP GET
#using the IRI in FROM clause as Data Source URL

query="DEFINE get:soft \"replace\"
SELECT DISTINCT * FROM <#{dsn}> WHERE {?s ?p ?o} "

#Assume use of local installation of Virtuoso 
#otherwise you can change URL to that of a public endpoint
#for example DBpedia: http://dbpedia.org/sparql

data=sparqlQuery(query, "http://localhost:8890/sparql/")

puts "Got data:"
p data

#
# End

Output

Got data:
[["s", "p", "o"], 
  ["http://dbpedia.org/resource/DBpedia", 
   "http://www.w3.org/1999/02/22-rdf-syntax-ns#type", 
   "http://www.w3.org/2002/07/owl#Thing"], 
  ["http://dbpedia.org/resource/DBpedia", 
   "http://www.w3.org/1999/02/22-rdf-syntax-ns#type", 
   "http://dbpedia.org/ontology/Work"], 
  ["http://dbpedia.org/resource/DBpedia", 
   "http://www.w3.org/1999/02/22-rdf-syntax-ns#type", 
   "http://dbpedia.org/class/yago/Software106566077"],
...

Conclusion

CSV was chosen over XML (re. output format) since this is about a "no-brainer installation and utilization" guide for a Ruby developer that already knows how to use Ruby for HTTP based data access. SPARQL just provides an added bonus to URL dexterity (delivered via URI abstraction) with regards to constructing Data Source Names or Addresses.

Related

• SPARQL and Ruby SPARQL Client Library Example
• Simple Guide for using SPARQL with Virtuoso
• General SPARQL Tutorial Collection
• Virtuoso Specific SPARQL Tutorial Collection
• The URI, URL, and Linked Data Meme's Generic HTTP URI.

1. You encounter DBpedia or the LOD Cloud Pictorial.
2. You look around (typically following your nose from link to link).
3. You attempt to publish your own stuff.
4. You get stuck.

The problems typically take the following form:

1. Functionality confusion about the complementary Name and Address functionality of a single URI abstraction
2. Terminology confusion due to conflation and over-loading of terms such as Resource, URL, Representation, Document, etc.
3. Inability to find robust tools with which to generate Linked Data from existing data sources such as relational databases, CSV files, XML, Web Services, etc.

To start addressing these problems, here is a simple guide for generating and publishing Linked Data using Virtuoso.

Step 1 - RDF Data Generation

Existing RDF data can be added to the Virtuoso RDF Quad Store via a variety of built-in data loader utilities.

Many options allow you to easily and quickly generate RDF data from other data sources:

• Install the Sponger Bookmarklet for the URIBurner service. Bind this to your own SPARQL-compliant backend RDF database (in this scenario, your local Virtuoso instance), and then Sponge some HTTP-accessible resources.
• Convert relational DBMS data to RDF using the Virtuoso RDF Views Wizard.
• Starting with CSV files, you can
  • Place them at an HTTP-accessible location, and use the Virtuoso Sponger to convert them to RDF or;
  • Use the CVS import feature to import their content into Virtuoso's relational data engine; then use the built-in RDF Views Wizard as with other RDBMS data.
• Starting from XML files, you can
  • Use Virtuoso's inbuilt XSLT-Processor for manual XML to RDF/XML transformation or;
  • Leverage the Sponger Cartridge for GRDDL, if there is a transformation service associated with your XML data source, or;
  • Let the Sponger analyze the XML data source and make a best-effort transformation to RDF.

Step 2 - Linked Data Deployment

Install the Faceted Browser VAD package (fct_dav.vad) which delivers the following:

1. Faceted Browser Engine UI
2. Dynamic Hypermedia Resource Generator
   • delivers descriptor resources for every entity (data object) in the Native or Virtual Quad Stores
   • supports a broad array of output formats, including HTML+RDFa, RDF/XML, N3/Turtle, NTriples, RDF-JSON, OData+Atom, and OData+JSON.

Step 3 - Linked Data Consumption & Exploitation

Three simple steps allow you, your enterprise, and your customers to consume and exploit your newly deployed Linked Data --

1. Load a page like this in your browser: http://<cname>[:<port>]/describe/?uri=<entity-uri>
   • <cname>[:<port>] gets replaced by the host and port of your Virtuoso instance
   • <entity-uri> gets replaced by the URI you want to see described -- for instance, the URI of one of the resources you let the Sponger handle.
2. Follow the links presented in the descriptor page.
3. If you ever see a blank page with a hyperlink subject name in the About: section at the top of the page, simply add the parameter "&sp=1" to the URL in the browser's Address box, and hit [ENTER]. This will result in an "on the fly" resource retrieval, transformation, and descriptor page generation.
4. Use the navigator controls to page up and down the data associated with the "in scope" resource descriptor.

Related

• Sample Descriptor Page (what you see post completion of the steps in this post)
• What is Linked Data, really?
• Painless Linked Data Generation via URIBurner
• How To Load RDF Data Into Virtuoso (various methods)
• Virtuoso Bulk Loader Script for RDF
• Bulk Loader Script for CSV
• Wizard based generation of RDF based Linked Data from ODBC accessible Relational Databases

Linked Data offers everyone a Web-scale, Enterprise-grade mechanism for platform-independent creation, curation, access, and integration of data.

The fundamental steps to creating Linked Data are as follows:

1. Choose a Name Reference Mechanism — i.e., URIs.

2. Choose a Data Model with which to Structure your Data — minimally, you need a model which clearly distinguishes

   1. Subjects (also known as Entities)
   2. Subject Attributes (also known as Entity Attributes), and
   3. Attribute Values (also known as Subject Attribute Values or Entity Attribute Values).

3. Choose one or more Data Representation Syntaxes (also called Markup Languages or Data Formats) to use when creating Resources with Content based on your chosen Data Model. Some Syntaxes in common use today are HTML+RDFa, N3, Turtle, RDF/XML, TriX, XRDS, GData, OData, OpenGraph, and many others.

4. Choose a URI Scheme that facilitates binding Referenced Names to the Resources which will carry your Content -- your Structured Data.

5. Create Structured Data by using your chosen Name Reference Mechanism, your chosen Data Model, and your chosen Data Representation Syntax, as follows:

   1. Identify Subject(s) using Resolvable URI(s).
   2. Identify Subject Attribute(s) using Resolvable URI(s).
   3. Assign Attribute Values to Subject Attributes. These Values may be either Literals (e.g., STRINGs, BLOBs) or Resolvable URIs.

You can create Linked Data (hypermedia-based data representations) Resources from or for many things. Examples include: personal profiles, calendars, address books, blogs, photo albums; there are many, many more.

Related

1. Linked Data an Introduction -- simple introduction to Linked Data and its virtues
2. How Data Makes Corporations Dumb -- Jeff Jonas (IBM) interview
3. Hypermedia Types -- evolving information portal covering different aspects of Hypermedia resource types
4. URIBurner -- service that generates Linked Data from a plethora of heterogeneous data sources
5. Linked Data Meme -- TimbL design issues note about Linked Data
6. Data 3.0 Manifesto -- note about format agnostic Linked Data
7. DBpedia -- large Linked Data Hub
8. Linked Open Data Cloud -- collection of Linked Data Spaces
9. Linked Open Commerce Cloud -- commerce (clicks & mortar and/or clicks & clicks) oriented Linked Data Space
10. LOD Cloud Cache -- massive Linked Data Space hosting most of the LOD Cloud Datasets
11. LOD2 Initiative -- EU Co-Funded Project to develop global knowledge space from LOD
.

Linked Data offers everyone a Web-scale, Enterprise-grade mechanism for platform-independent creation, curation, access, and integration of data.

The fundamental steps to creating Linked Data are as follows:

1. Choose a Name Reference Mechanism — i.e., URIs.

2. Choose a Data Model with which to Structure your Data — minimally, you need a model which clearly distinguishes

   1. Subjects (also known as Entities)
   2. Subject Attributes (also known as Entity Attributes), and
   3. Attribute Values (also known as Subject Attribute Values or Entity Attribute Values).

3. Choose one or more Data Representation Syntaxes (also called Markup Languages or Data Formats) to use when creating Resources with Content based on your chosen Data Model. Some Syntaxes in common use today are HTML+RDFa, N3, Turtle, RDF/XML, TriX, XRDS, GData, and OData; there are many others.

4. Choose a URI Scheme that facilitates binding Referenced Names to the Resources which will carry your Content -- your Structured Data.

5. Create Structured Data by using your chosen Name Reference Mechanism, your chosen Data Model, and your chosen Data Representation Syntax, as follows:

   1. Identify Subject(s) using Resolvable URI(s).
   2. Identify Subject Attribute(s) using Resolvable URI(s).
   3. Assign Attribute Values to Subject Attributes. These Values may be either Literals (e.g., STRINGs, BLOBs) or Resolvable URIs.

You can create Linked Data (hypermedia-based data representations) Resources from or for many things. Examples include: personal profiles, calendars, address books, blogs, photo albums; there are many, many more.

Related

1. Hypermedia Types -- evolving information portal covering different aspects of Hypermedia resource types
2. URIBurner -- service that generates Linked Data from a plethora of heterogeneous data sources
3. Linked Data Meme -- TimbL design issues note about Linked Data
4. Data 3.0 Manifesto -- note about format agnostic Linked Data
5. DBpedia -- large Linked Data Hub
6. Linked Open Data Cloud -- collection of Linked Data Spaces
7. Linked Open Commerce Cloud -- commerce (clicks & mortar and/or clicks & clicks) oriented Linked Data Space
8. LOD Cloud Cache -- massive Linked Data Space hosting most of the LOD Cloud Datasets
9. LOD2 Initiative -- EU Co-Funded Project to develop global knowledge space from LOD
.

A service from OpenLink Software, available at: http://uriburner.com, that enables anyone to generate structured descriptions -on the fly- for resources that are already published to HTTP based networks. These descriptions exist as hypermedia resource representations where links are used to identify:

• the entity (data object or datum) being described,
• each of its attributes, and
• each of its attributes values (optionally).

The hypermedia resource representation outlined above is what is commonly known as an Entity-Attribute-Value (EAV) Graph. The use of generic HTTP scheme based Identifiers is what distinguishes this type of hypermedia resource from others.

Why is it Important?

The virtues (dual pronged serendipitous discovery) of publishing HTTP based Linked Data across public (World Wide Web) or private (Intranets and/or Extranets) is rapidly becoming clearer to everyone. That said, the nuance laced nature of Linked Data publishing presents significant challenges to most. Thus, for Linked Data to really blossom the process of publishing needs to be simplified i.e., "just click and go" (for human interaction) or REST-ful orchestration of HTTP CRUD (Create, Read, Update, Delete) operations between Client Applications and Linked Data Servers.

How Do I Use It?

In similar vane to the role played by FeedBurner with regards to Atom and RSS feed generation, during the early stages of the Blogosphere, it enables anyone to publish Linked Data bearing hypermedia resources on an HTTP network. Thus, its usage covers two profiles: Content Publisher and Content Consumer.

Content Publisher

The steps that follow cover all you need to do:

• place a tag within your HTTP based hypermedia resource (e.g. within section for HTML )
• use a URL via the @href attribute value to identify the location of the structured description of your resource, in this case it takes the form: http://linkeddata.uriburner.com/about/id/{scheme-or-protocol}/{your-hostname-or-authority}/{your-local-resource}
• for human visibility you may consider adding associating a button (as you do with Atom and RSS) with the URL above.

That's it! The discoverability (SDQ) of your content has just multiplied significantly, its structured description is now part of the Linked Data Cloud with a reference back to your site (which is now a bona fide HTTP based Linked Data Space).

Examples

HTML+RDFa based representation of a structured resource description:

<link rel="describedby" title="Resource Description (HTML)"type="text/html" href="http://linkeddata.uriburner.com/about/id/http/example.org/xyz.html"/>

JSON based representation of a structured resource description:

<link rel="describedby" title="Resource Description (JSON)" type="application/json" href="http://linkeddata.uriburner.com/about/id/http/example.org/xyz.html"/>

N3 based representation of a structured resource description:

<link rel="describedby" title="Resource Description (N3)" type="text/n3" href="http://linkeddata.uriburner.com/about/id/http/example.org/xyz.html"/>

RDF/XML based representations of a structured resource description:

<link rel="describedby" title="Resource Description (RDF/XML)" type="application/rdf+xml" href="http://linkeddata.uriburner.com/about/id/http/example.org/xyz.html"/>

Content Consumer

As an end-user, obtaining a structured description of any resource published to an HTTP network boils down to the following steps:

1. go to: http://uriburner.com
2. drag the Page Metadata Bookmarklet link to your Browser's toolbar
3. whenever you encounter a resource of interest (e.g. an HTML page) simply click on the Bookmarklet
4. you will be presented with an HTML representation of a structured resource description (i.e., identifier of the entity being described, its attributes, and its attribute values will be clearly presented).

Examples

• Description of a Book culled from an Amazon web page
• Description of a product offering culled from a BestBuy web page
• Description of a product (a camera) culled from a CNET web page
• Description of the same CNET product as an Offer on eBay (exposed by the description above via seeAlso property value).

If you are a developer, you can simply perform an HTTP operation request (from your development environment of choice) using any of the URL patterns presented below:

• curl -I -H "Accept: text/html" http://linkeddata.uriburner.com/about/id/{scheme}/{authority}/{local-path}

JSON:

• curl -I -H "Accept: application/json" http://linkeddata.uriburner.com/about/id/{scheme}/{authority}/{local-path}
• curl http://linkeddata.uriburner.com/about/data/json/{scheme}/{authority}/{local-path}

Notation 3 (N3):

• curl -I -H "Accept: text/n3" http://linkeddata.uriburner.com/about/id/{scheme}/{authority}/{local-path}
• curl http://linkeddata.uriburner.com/about/data/n3/{scheme}/{authority}/{local-path}

• curl -I -H "Accept: text/turtle" http://linkeddata.uriburner.com/about/id/{scheme}/{authority}/{local-path}
• curl http://linkeddata.uriburner.com/about/data/ttl/{scheme}/{authority}/{local-path}

RDF/XML:

• curl -I -H "Accept: application/rdf+xml" http://linkeddata.uriburner.com/about/id/{scheme}/{authority}/{local-path}
• curl http://linkeddata.uriburner.com/about/data/xml/{scheme}/{authority}/{local-path}

Conclusion

URIBurner is a "deceptively simple" solution for cost-effective exploitation of HTTP based Linked Data meshes. It doesn't require any programming or customization en route to immediately realizing its virtues.

If you like what URIBurner offers, but prefer to leverage its capabilities within your domain -- such that resource description URLs reside in your domain, all you have to do is perform the following steps:

1. download a copy of Virtuoso (for local desktop, workgroup, or data center installation) or
2. instantiate Virtuoso via the Amazon EC2 Cloud
3. enable the Sponger Middleware component via the RDF Mapper VAD package (which includes cartridges for over 30 different resources types)

When you install your own URIBurner instances, you also have the ability to perform customizations that increase resource description fidelity in line with your specific needs. All you need to do is develop a custom extractor cartridge and/or meta cartridge.

Related:

• Virtuoso Sponger Middleware -- (technology behind URIBurner Service)
• Animation demonstrating how the Virtuoso Sponger works.

At the end of the marathon session, it was clear to me that a blog post was required for future reference, at the very least :-)

What is Linked Data?

"Data Access by Reference" mechanism for Data Objects (or Entities) on HTTP networks. It enables you to Identify a Data Object and Access its structured Data Representation via a single Generic HTTP scheme based Identifier (HTTP URI). Data Object representation formats may vary; but in all cases, they are hypermedia oriented, fully structured, and negotiable within the context of a client-server message exchange.

Why is it Important?

Information makes the world tick!

Information doesn't exist without data to contextualize.

Information is inaccessible without a projection (presentation) medium.

All information (without exception, when produced by humans) is subjective. Thus, to truly maximize the innate heterogeneity of collective human intelligence, loose coupling of our information and associated data sources is imperative.

How is Linked Data Delivered?

Linked Data is exposed to HTTP networks (e.g. World Wide Web) via hypermedia resources bearing structured representations of data object descriptions. Remember, you have a single Identifier abstraction (generic HTTP URI) that embodies: Data Object Name and Data Representation Location (aka URL).

How are Linked Data Object Representations Structured?

A structured representation of data exists when an Entity (Datum), its Attributes, and its Attribute Values are clearly discernible. In the case of a Linked Data Object, structured descriptions take the form of a hypermedia based Entity-Attribute-Value (EAV) graph pictorial -- where each Entity, its Attributes, and its Attribute Values (optionally) are identified using Generic HTTP URIs.

Examples of structured data representation formats (content types) associated with Linked Data Objects include:

• text/html
• text/turtle
• text/n3
• application/json
• application/rdf+xml
• Others

How Do I Create Linked Data oriented Hypermedia Resources?

You markup resources by expressing distinct entity-attribute-value statements (basically these a 3-tuple records) using a variety of notations:

• (X)HTML+RDFa,
• JSON,
• Turtle,
• N3,
• TriX,
• TriG,
• RDF/XML, and
• Others (for instance you can use Atom data format extensions to model EAV graph as per OData initiative from Microsoft).

You can achieve this task using any of the following approaches:

• Notepad
• WYSIWYG Editor
• Transformation of Database Records via Middleware
• Transformation of XML based Web Services output via Middleware
• Transformation of other Hypermedia Resources via Middleware
• Transformation of non Hypermedia Resources via Middleware
• Use a platform that delivers all of the above.

Practical Examples of Linked Data Objects Enable

• Describe Who You Are, What You Offer, and What You Need via your structured profile, then leave your HTTP network to perform the REST (serendipitous discovery of relevant things)
• Identify (via map overlay) all items of interest based on a 2km+ radious of my current location (this could include vendor offerings or services sought by existing or future customers)
• Share the latest and greatest family photos with family members *only* without forcing them to signup for Yet Another Web 2.0 service or Social Network
• No repetitive signup and username and password based login sequences per Web 2.0 or Mobile Application combo
• Going beyond imprecise Keyword Search to the new frontier of Precision Find - Example, Find Data Objects associated with the keywords: Tiger, while enabling the seeker disambiguate across the "Who", "What", "Where", "When" dimensions (with negation capability)
• Determine how two Data Objects are Connected - person to person, person to subject matter etc. (LinkedIn outside the walled garden)
• Use any resource address (e.g blog or bookmark URL) as the conduit into a Data Object mesh that exposes all associated Entities and their social network relationships
• Apply patterns (social dimensions) above to traditional enterprise data sources in combination (optionally) with external data without compromising security etc.

How Do OpenLink Software Products Enable Linked Data Exploitation?

Our data access middleware heritage (which spans 16+ years) has enabled us to assemble a rich portfolio of coherently integrated products that enable cost-effective evaluation and utilization of Linked Data, without writing a single line of code, or exposing you to the hidden, but extensive admin and configuration costs. Post installation, the benefits of Linked Data simply materialize (along the lines described above).

Our main Linked Data oriented products include:

• OpenLink Data Explorer -- visualizes Linked Data or Linked Data transformed "on the fly" from hypermedia and non hypermedia data sources
• URIBurner -- a "deceptively simple" solution that enables the generation of Linked Data "on the fly" from a broad collection of data sources and resource types
• OpenLink Data Spaces -- a platform for enterprises and individuals that enhances distributed collaboration via Linked Data driven virtualization of data across its native and/or 3rd party content manager for: Blogs, Wikis, Shared Bookmarks, Discussion Forums, Social Networks etc
• OpenLink Virtuoso -- a secure and high-performance native hybrid data server (Relational, RDF-Graph, Document models) that includes in-built Linked Data transformation middleware (aka. Sponger).

Related

• Hypertext Transfer Protocol 1.1 RFC
• Open Data Protocol Glossary
• Simple Explanation of RDF and Linked Data Dynamics
• Collection of post from the past about Linked Data
• Are We There Yet Re. Web++? -- includes link to podcast conversation with Jon Udell
• Web of Linked Data Pivoting Demo from TED -- by Microsoft's Gary Flake
• Microsoft Pivot atop Virtuoso Quad Store's Faceted Browser Engine-- My Demonstration of EAV model transcending data representation variations (i.e., RDF's EAV data model data served up in Microsoft CXML data representation format).

In this post I provide a brief re-introduction to this essential aspect of Virtuoso.

What is it?

This component of Virtuoso is known as the Virtual Database Engine (VDBMS). It provides transparent high-performance and secure access to disparate data sources that are external to Virtuoso. It enables federated access and integration of data hosted by any ODBC- or JDBC-accessible RDBMS, RDF Store, XML database, or Document (Free Text)-oriented Content Management System. In addition, it facilitates integration with Web Services (SOAP-based SOA RPCs or REST-fully accessible Web Resources).

Why is it important?

In the most basic sense, you shouldn't need to upgrade your existing database engine version simply because your current DBMS and Data Access Driver combo isn't compatible with ODBC-compliant desktop tools such as Microsoft Access, Crystal Reports, BusinessObjects, Impromptu, or other of ODBC, JDBC, ADO.NET, or OLE DB-compliant applications. Simply place Virtuoso in front of your so-called "legacy database," and let it deliver the compliance levels sought by these tools

In addition, it's important to note that today's enterprise, through application evolution, company mergers, or acquisitions, is often faced with disparately-structured data residing in any number of line-of-business-oriented data silos. Compounding the problem is the exponential growth of user-generated data via new social media-oriented collaboration tools and platforms. For companies to cost-effectively harness the opportunities accorded by the increasing intersection between line-of-business applications and social media, virtualization of data silos must be achieved, and this virtualization must be delivered in a manner that doesn't prohibitively compromise performance or completely undermine security at either the enterprise or personal level. Again, this is what you get by simply installing Virtuoso.

How do I use it?

The VDBMS may be used in a variety of ways, depending on the data access and integration task at hand. Examples include:

Relational Database Federation

You can make a single ODBC, JDBC, ADO.NET, OLE DB, or XMLA connection to multiple ODBC- or JDBC-accessible RDBMS data sources, concurrently, with the ability to perform intelligent distributed joins against externally-hosted database tables. For instance, you can join internal human resources data against internal sales and external stock market data, even when the HR team uses Oracle, the Sales team uses Informix, and the Stock Market figures come from Ingres!

Conceptual Level Data Access using the RDF Model

You can construct RDF Model-based Conceptual Views atop Relational Data Sources. This is about generating HTTP-based Entity-Attribute-Value (E-A-V) graphs using data culled "on the fly" from native or external data sources (Relational Tables/Views, XML-based Web Services, or User Defined Types).

You can also derive RDF Model-based Conceptual Views from Web Resource transformations "on the fly" -- the Virtuoso Sponger (RDFizing middleware component) enables you to generate RDF Model Linked Data via a RESTful Web Service or within the process pipeline of the SPARQL query engine (i.e., you simply use the URL of a Web Resource in the FROM clause of a SPARQL query).

It's important to note that Views take the form of HTTP links that serve as both Data Source Names and Data Source Addresses. This enables you to query and explore relationships across entities (i.e., People, Places, and other Real World Things) via HTTP clients (e.g., Web Browsers) or directly via SPARQL Query Language constructs transmitted over HTTP.

Conceptual Level Data Access using ADO.NET Entity Frameworks

As an alternative to RDF, Virtuoso can expose ADO.NET Entity Frameworks-based Conceptual Views over Relational Data Sources. It achieves this by generating Entity Relationship graphs via its native ADO.NET Provider, exposing all externally attached ODBC- and JDBC-accessible data sources. In addition, the ADO.NET Provider supports direct access to Virtuoso's native RDF database engine, eliminating the need for resource intensive Entity Frameworks model transformations.

Related

• Attaching ODBC or JDBC accessible Relational Tables to Virtuoso
• Using an HTML based Wizard to Generate RDF based Linked Views over Relational Tables
• Screencast Demonstrating Wizard based generation of RDF based Linked Data Views Part 1
• Screencast Demonstrating Wizard based generation of RDF based Linked Data Views Part 1
• Generating RDF based Linked Data from non RDF based Web Resources via the Sponger
• Building ADO.NET based Entity Frameworks Views over Relational Data
• Building Silverlight Rich Internat Applicaitons using ADO.NET, Entity Frameworks, and RDF based Linked Data.

When trying to understand HTTP based Linked Data, especially if you're well versed in DBMS technology use (User, Power User, Architect, Analyst, DBA, or Programmer) think:

• Open Database Connectivity (ODBC) without operating system, data model, or wire-protocol specificity or lock-in potential
• Java Database Connectivity (JDBC) without programming language specificity
• ADO.NET without .NET runtime specificity and .NET bound language specificity
• OLE-DB without Windows operating system & programming language specificity
• XMLA without XML format specificity - with Tabular and Multidimensional results formats expressible in a variety of data representation formats.
• All of the above scoped to the Record rather than Container level, with Generic HTTP scheme URIs associated with each Record, Field, and Field value (optionally)

Remember the need for Data Access & Integration technology is the by product of the following realities:

1. Human curated data is ultimately dirty, because:
   • our thick thumbs, inattention, distractions, and general discomfort with typing, make typos prevalent
   • database engines exist for a variety of data models - Graph, Relational, Hierarchical;
   • within databases you have different record container/partition names e.g. Table Names;
   • within a database record container you have records that are really aspects of the same thing (different keys exist in a plethora of operational / line of business systems that expose aspects of the same entity e.g., customer data that spans Accounts, CRM, ERP application databases);
   • different field names (one database has "EMP" while another has "Employee") for the same record
   .
2. Units of measurement is driven by locale, the UK office wants to see sales in Pounds Sterling while the French office prefers Euros etc.
3. All of the above is subject to context halos which can be quite granular re. sensitivity e.g. staff travel between locations that alter locales and their roles; basically, profiles matters a lot.

Related

• ODBC and WODBC (Web Open Database Connectivity) Comparison
• Creating, Deploying, and Exploiting Linked Data Presentation
• Open Data Protocol Project

1. Acquire your own personal or service specific data space in the Cloud. Think DBase, Paradox, FoxPRO, Access of yore, but with the power of Oracle, Informix, Microsoft SQL Server etc.. using a Conceptual, as opposed to solely Logical, model based DBMS (i.e., a Hybrid DBMS Engine for: SQL, RDF, XML, and Full Text)
2. Ability to share and control access to your resources using innovations like FOAF+SSL, OpenID, and OAuth, all from one place
3. Construction of personal or organization based FOAF profiles in a matter of minutes; by simply creating a basic DBMS (or ODS application layer) account; and then using this profile to create strong links (references) to all your Data silos (esp. those from the Web 2.0 realm)
4. Load data sets from the LOD cloud or Sponge existing Web resources (i.e., on the fly data transformation to RDF model based Linked Data) and then use the combination to build powerful lookup services that enrich the value of URLs (think: Web addressable reports holding query results) that you publish
5. Bind all of the above to a domain that you own (e.g. a .Name domain) so that you have an attribution-friendly "authority" component for resource URLs and Entity URIs published from your Personal Linked Data Space on the Web (or private HTTP network).

In a nutshell, the AWS Cloud infrastructure simplifies the process of generating Federated presence on the Internet and/or World Wide Web. Remember, centralized networking models always end up creating data silos, in some context, ultimately! :-)

As the "Linked Data" meme has gained momentum you've more than likely been on the receiving end of dialog with Linked Open Data community members (myself included) that goes something like this:

"Do you have a URI", "Get yourself a URI", "Give me a de-referencable URI" etc..

And each time, you respond with a URL -- which to the best of your Web knowledge is a bona fide URI. But to your utter confusion you are told: Nah! You gave me a Document URI instead of the URI of a real-world thing or object etc..

What's up with that?

Well our everyday use of the Web is an unfortunate conflation of two distinct things, which have Identity: Real World Objects (RWOs) & Address/Location of Documents (Information bearing Resources).

The "Linked Data" meme is about enhancing the Web by unobtrusively reintroducing its core essence: the generic HTTP URI, a vital piece of Web Architecture DNA. Basically, its about so realizing the full capabilities of the Web as a platform for Open Data Identification, Definition, Access, Storage, Representation, Presentation, and Integration.

What is a Real World Object?

People, Places, Music, Books, Cars, Ideas, Emotions etc..

What is a URI?

A Uniform Resource Identifier. A global identifier mechanism for network addressable data items. Its sole function is Name oriented Identification.

URI Generic Syntax

The constituent parts of a URI (from URI Generic Syntax RFC) are depicted below:

What is a URL?

A location oriented HTTP scheme based URI. The HTTP scheme introduces a powerful and inherent duality that delivers:

1. Resource Address/Location Identifier
2. Data Access mechanism for an Information bearing Resource (Document, File etc..)

So far so good!

What is an HTTP based URI?

The kind of URI Linked Data aficionados mean when they use the term: URI.

An HTTP URI is an HTTP scheme based URI. Unlike a URL, this kind of HTTP scheme URI is devoid of any Web Location orientation or specificity. Thus, Its inherent duality provides a more powerful level of abstraction. Hence, you can use this form of URI to assign Names/Identifiers to Real World Objects (RWO). Even better, courtesy of the Identity/Address duality of the HTTP scheme, a single URI can deliver the following:

1. RWO Identfier/Name
2. RWO Metadata document Locator (courtesy of URL aspect)
3. Negotiable Representation of the Located Document (courtesy of HTTP's content negotiation feature).

What is Metadata?

Data about Data. Put differently, data that describes other data in a structured manner.

How Do we Model Metadata?

The predominant model for metadata is the Entity-Attribute-Value + Classes & Relationships model (EAV/CR). A model that's been with us since the inception of modern computing (long before the Web).

What about RDF?

The Resource Description Framework (RDF) is a framework for describing Web addressable resources. In a nutshell, its a framework for adding Metadata bearing Information Resources to the current Web. Its comprised of:

1. Entity-Attribute-Value (aka. Subject-Predictate-Object) plus Classes & Relationships (Data Dictionaries e.g., OWL) metadata model
2. A plethora of instance data representation formats that include: RDFa (when doing so within (X)HTML docs), Turtle, N3, TriX, RDF/XML etc.

What's the Problem Today?

The ubiquitous use of the Web is primarily focused on a Linked Mesh of Information bearing Documents. URLs rather than generic HTTP URIs are the prime mechanism for Web tapestry; basically, we use URLs to conduct Information -- which is inherently subjective -- instead of using HTTP URIs to conduct "Raw Data" -- which is inherently objective.

Note: Information is "data in context", it isn't the same thing as "Raw Data". Thus, if we can link to Information via the Web, why shouldn't we be able to do the same for "Raw Data"?

How Does the Link Data meme solve the problem?

The meme simply provides a set of guidelines (best practices) for producing Web architecture friendly metadata. Meaning: when producing EAV/CR model based metadata, endow Subjects, their Attributes, and Attribute Values (optionally) with HTTP URIs. By doing so, a new level of Link Abstraction on the Web is possible i.e., "Data Item to Data Item" level links (aka hyperdata links). Even better, when you de-reference a RWO hyperdata link you end up with a negotiated representations of its metadata.

Conclusion

Linked Data is ultimately about an HTTP URI for each item in the Data Organization Hierarchy :-)

Related

1. History of how "Resource" became part of URI - historic account by TimBL
2. Linked Data Design Issues Document - TimBL's initial Linked Data Guide
3. Linked Data Rules Simplified - My attempt at simplifying the Linked Data Meme without SPARQL & RDF distraction
4. Linked Data & Identity - another related post
5. The Linked Data Meme's Value Proposition
6. So What Does "HREF" stand for anyway?
7. My Del.icio.us hosted Bookmark Data Space for Identity Schemes
8. TimBL's Ted Talk re. "Raw Linked Data"
9. Resource Oriented Architecture
10. More Famous Than Simon Cowell .

1. Identify or Name things using HTTP URIs
2. Describe things using the RDF metadata model
3. Increase link data mesh density on the Web by linking (referring) to things in other data spaces using their HTTP URIs.

If you perform the steps above, on any HTTP network (e.g. World Wide Web), you implicitly bind the Names/Identifiers of things to negotiable representations of their metadata (description) bearing documents.

Also note, you can create and deploy the resulting RDF metadata using any of the following approaches:

1. RDFa within (X)HTML documents
2. N3, Turtle, TriX, RDF/XML etc. based documents
3. Programmatically generated variants of 1&2.

Related

• What is the Linked Data meme about?
• Simple Explanation of RDF and Linked Data Dynamics

Dr. Dre is one of the artists in the Linked Data Space we host for the BBC. He is also referenced in music oriented data spaces such as DBpedia, MusicBrainz and Last.FM (to name a few).

Challenge:

How do I obtain a holistic view of the entity "Dr. Dre" across the BBC, MusicBrainz, and Last.FM data spaces? We know the BBC published Linked Data, but what about Last.FM and MusicBrainz? Both of these data spaces only expose XML or JSON data via REST APIs?

Solution:

Steps:

1. Go to Last.FM and search using pattern: Dr. Dre (you will end up with this URL: http://www.last.fm/music/Dr.+Dre)
2. Go to the Virtuoso powered BBC Linked Data Space home page and enter: http://bbc.openlinksw.com/about/html/http://www.last.fm/music/Dr.+Dre
3. Go to the BBC Linked Data Space home page and type full text pattern (using default tab): Dr. Dre, then view Dr. Dre's metadata via the Statistics Link.

What Happened?

The following took place:

1. Virtuoso Sponger sent an HTTP GET to Last.FM
2. Distilled the "Artist" entity "Dr. Dre" from the page, and made a Linked Data graph
3. Inverse Functional Property and sameAs reasoning handled the Meshup (augmented graph from a conjunctive query processing pipeline)
4. Links for "Dr. Dre" across BBC (sameAs), Last.FM (seeAlso), via DBpedia URI.

The new enhanced URI for Dr. Dre now provides a rich holistic view of the aforementioned "Artist" entity. This URI is usable anywhere on the Web for Linked Data Conduction :-)

Related (as in NearBy)

• Augmenting Last.fm Data with BBC data on the Talis Platform

section from IETF's Domain Keys spec. (paraphrased by me)

.

The Linked Data meme is based on the use of HTTP based URIs as reference / identifier labels associated with the "identity abstraction" referred to above. Thus, when you de-reference (request information about) an HTTP based URI you ultimately end up with a resource URL that exposes the "constellation of characteristics" mentioned above, in a representation negotiated at request time -- between an HTTP client and server e.g., (X)HTML, JSON, XML, RDF/XML, N3, Turtle, Trix, others :-)

Related

• What is the Linked Data meme About?
• Simple Explanation of RDF & Linked Data Dynamics.
• Handle -- Internet wide Identity Scheme and Resolution System

Note: In proper Web parlance, a data object is referred to as a resource.

Simple example (using DBpedia)

In the Linked Data realm, If you want to make a reference to the Linked Data meme in a blog post, you are better off using the resource URI: http://dbpedia.org/resource/Linked_Data, instead of the Web page URL: http://dbpedia.org/page/Linked_Data, which is the address of a physical document (an information conveying artifact) that at best visually presents the negotiated representation of a resource description.

Why is this valuable?

In the simplest sense, you only have one focal point for referencing (referring to) and de-referencing (retrieving data about) a given Web resource. It protects you from the impact of Web document location changes (amongst many other things).

Remember, a single URI is a conduit into a realm where the identity, access, representation, presentation, and storage of a resource (data object) are completely distinct. It's the mechanism for conducting data across network, machine, operating system, dbms engine, application, and service (API) boundaries. Thus, without "linked data meme" prescribed URI referencing and de-referencing, we are simply back to "business as usual" re. the industry at large, where networks, operating systems, dbms engines, applications, and services (APIs) become the basis for "data lock-in" and silo construction.

Going forward

Take a second to think about the profound virtues of the ubiquitous Web of Linked Document URLs that we have today, and then apply that thinking to the burgeoning Web of Linked Data URIs, that has just turned corner and heading in everyone's direction at full blast.

Note to "Social Media" players: Who you know isn't the canonical object of sociality. What you are i.e., your description and the data objects it exposes, are real objects of your sociality :-)

Related

• Other post in this Blog Data Space associated with "Linked Data".

The acronym stands for: Resource Description Framework. And that's just what it is.

RDF is comprised of a Data Model (EAV/CR Graph) and Data Representation Formats such as: N3, Turtle, RDF/XML etc.

RDF's essence is about: "Entities" and "Attributes" being URI based, while "Values" may be URI or Literals (typed or untyped) based.

URIs are Entity Identifiers.

What is Linked Data?

Short for "Web of Linked Data" or "Linked Data Web".

A term coined by TimBL that describes an HTTP based "data access by reference pattern" that uses a single pointer or handle for "referring to" and "obtaining actual data about" an entity.

Linked Data uses the deceptively simple messaging scheme of HTTP to deliver a granular entity reference and access mechanism that transcends traditional computing boundaries such as: operating system, application, database engines, and networks.

How are Linked Data & RDF Related?

Linked Data simply mandates the following re. RDF:

• URIs should be HTTP based so that you can "refer to" (Reference) an Entity, its Attributes, or URI based Attribute values via the Web (infact any HTTP based network e.g., Intranets and Extranets)
• URIs should also be HTTP based so that you can use them to de-reference resource descriptions via the Web (or Intranets and Extranets).

Note: by Entity I am also referring to: a resource (Web parlance), data item, data object, real-world object, or datum.

Linked Data is also about, using URIs and HTTP's content negotiation feature to separate: presentation, representation, access, and identity of data items. Even better, content negotiation can be driven by user agent and/or data server based quality of service algorithms (representation preference order schemes).

To conclude, Linked Data is ultimately about the realization that: Data is the new Electricity, and it's conductors are URIs :-)

Tip to governments of the world: we are in exponential times, the current downturn is but one side of the "exponential times ledger", the other side of the "exponential times ledger" is simply about unleashing "raw data" -- in structured form -- into the Web, so that "citizen analysts" can blossom and ultimately deliver the transparency desperately sought at every level of the economic value chain. Think: "raw data ready" whenever you ponder about "shovel ready" infrastructure projects!

Your Life, Profession, Web, and Internet do not need to become mutually exclusive due to "information overload".

Solution:

A platform or service that delivers a point of online presence that embodies the fundamental separation of: Identity, Data Access, Data Representation, Data Presentation, by adhering to Web and Internet protocols.

How:

Typical post installation (Local or Cloud) task sequence:

1. Identify myself (happens automatically by way of registration)
2. If in an LDAP environment, import accounts or associate system with LDAP for account lookup and authentication
3. Identify Online Accounts (by fleshing out profile) which also connects system to online accounts and their data
4. Use Profile for granular description (Biography, Interests, WishList, OfferList, etc.)
5. Optionally upstream or downstream data to and from my online accounts
6. Create content Tagging Rules
7. Create rules for associating Tags with formal URIs
8. Create automatic Hyperlinking Rules for reuse when new content is created (e.g. Blog posts)
9. Exploit Data Portability virtues of RSS, Atom, OPML, RDFa, RDF/XML, and other formats for imports and exports
10. Automatically tag imported content
11. Use function-specific helper application UIs for domain specific data generation e.g. AddressBook (optionally use vCard import), Calendar (optionally use iCalendar import), Email, File Storage (use WebDAV mount with copy and paste or HTTP GET), Feed Subscriptions (optionally import RSS/Atom/OPML feeds), Bookmarking (optionally import bookmark.html or XBEL) etc..
12. Optionally enable "Conversation" feature (today: Social Media feature) across the relevant application domains (manage conversations under covers using NNTP, the standard for this functionality realm)
13. Generate HTTP based Entity IDs (URIs) for every piece of data in this burgeoning data space
14. Use REST based APIs to perform CRUD tasks against my data (local and remote) (SPARQL, GData, Ubiquity Commands, Atom Publishing)
15. Use OpenID, OAuth, FOAF+SSL, FOAF+SSL+OpenID for accessing data elsewhere
16. Use OpenID, OAuth, FOAF+SSL, FOAF+SSL+OpenID for Controlling access to my data (Self Signed Certificate Generation, Browser Import of said Certificate & associated Private Key, plus persistence of Certificate to FOAF based profile data space in "one click")
17. Have a simple UI for Entity-Attribute-Value or Subject-Predicate-Object arbitrary data annotations and creation since you can't pre model an "Open World" where the only constant is data flow
18. Have my Personal URI (Web ID) as the single entry point for controlled access to my HTTP accessible data space

I've just outlined a snippet of the capabilities of the OpenLink Data Spaces platform. A platform built using OpenLink Virtuoso, architected to deliver: open, platform independent, multi-model, data access and data management across heterogeneous data sources.

All you need to remember is your URI when seeking to interact with your data space.

Related

1. Get Yourself a URI (Web ID) in 5 Minutes or Less!
2. Various posts over the years about Data Spaces
3. Future of Desktop Post
4. Simplify My Life Post by Bengee Nowack

At the current time we have loaded 100% of all the very large data sets from the LOD Cloud. As result, we can start the process of exposing Linked Data virtues in a manner that's palatable to users, developers, and database professionals across the Web 1.0, 2.0, and 3.0 spectrums.

What does this mean?

You can use the "Search & Find" or"URI Lookup" or SPARQL endpoint associated with the LOD cloud hosting instance to perform the following tasks:

1. Find entities associated with full text search patterns -- Google Style, but with Entity & Text proximity Rank instead of Page Rank, since we are dealing with Entities rather than documents about entities
2. Find and Lookup entities by Identifier (URI) -- which is helpful when locating URIs to use for identify entities in your own linked data spaces on the Web
3. View entity descriptions via a variety of representation formats (HTML, RDFa, RDF/XML, N3, Turtle etc.)
4. Determine uses of entity identifiers across the LOD cloud -- which helps you select preferred URIs based on usage statistics.

What does it offer Web 1.0 and 2.0 developers?

If you don't want to use the SPARQL based Web Service, or other Linked Data Web oriented APIs for interacting with the LOD cloud programmatically, you can simply use the powerful REST style Web Service that provides URL parameters for performing full text oriented "Search", entity oriented "Find" queries, and faceted navigation over the huge data corpus with results data returned in JSON and XML formats.

Next Steps:

Amazon have agreed to add all the LOD Cloud data sets to their existing public data sets collective. Thus, the data sets we are loading will be available in "raw data" (RDF) format on the public data sets page via Named Elastic Block Storage (EBS) Snapshots); meaning, you can make an EC2 AMI (e.g. a Linux, Windows, Solaris) and install an RDF quad or triple store of choice into your AMI, then simply load data from the LOD cloud based on your needs.

In addition to the above, we are also going to offer a Virtuoso 6.0 Cluster Edition based LOD Cloud AMI (as we've already done with DBpedia, MusicBrainz, NeuroCommons, and Bio2Rdf) that will enable you to simply instantiate a personal and service specific edition of Virtuoso with all the LOD data in place and fully tuned for performance and scalability; basically, you will simply press "Instantiate AMI" and a LOD cloud data space, in true Linked Data from, will be at your disposal within minutes (i.e. the time it takes the DB to start).

Work on the migration of the LOD data to EC2 starts this week. Thus, if you are interested in contributing an RDF based data set to the LOD cloud now is the time to get your archive links in place on the (see: ESW Wiki page for LOD Data Sets).

Naturally, I am not expecting everyone to agree with me. I am simply making my contribution to what will remain facinating discourse for a long time to come :-)

Related

• Web 3.0 The Best Official Definition Imaginable -- Nova Spivack's

Today, I revisited the same article -- and to my shock and horror -- my comments do not exist (note: the site did accept my comments yesterday!). Even more frustrating for me, I now have to expend time I don't have re-writing my comments due to the depth and danger of the inaccuracies in this post re. RDF in general.

Important Note to ebiz and David:

Please look into what happened to my comments. It's too early for me to conclude that subjective censorship is a play on the Web -- which isn't a hard copy journalistic format style of platform where editors get away with such shenanigans. The Web is a sticky database, and outer joining is well and truly functional (meaning: exclusion and omission ultimately come back to bite via full outer join query results against the Web DB).

By the way, if you publish the comments I made to the post (yesterday), I will add a note to this post, accordingly.

Yes! David just confirmed to me via Twitter that this is yet another comment system related issue and absolutely no intent to censor etc. His words Twervatim :-)

For sake of clarity, I've itemized the inaccuracies and applied my correction comments (inline) accordingly:

Inaccuracy #1:

Resource Description Framework (RDF), a part of the XML story, provides interoperability between applications that exchange information.

Correction #1:

RDF and XML are not inextricably linked in any way. RDF is part Data Model (EAV/CR style Graph) with associated markup and data serialization formats that include: N3, Turtle, TriX, RDF/XML etc.

Inaccuracy #2:

RDF uses XML to define a foundation for processing metadata and to provide a standard metadata infrastructure for both the Web and the enterprise.

Correction #2:

RDF/XML is an XML based markup and data serialization format. As a markup language it can be used for creating RDF model records/statements (using Subject, Predicate, Object or Entity, Attribute, Value). As a serialization format, it provides a mechanism for marshaling RDF data across data managers and data consumers.

Inaccuracy #3:

The difference between the two is that XML is used to transport data using a common format, while RDF is layered on top of XML defining a broad category of data.

Correction #3:

See earlier corrections above.

Inaccuracy #4:

When the XML data is declared to be of the RDF format, applications are then able to understand the data without understanding who sent it.

Correction #4:

You do not declare data to be of RDF format. RDF isn't a format it is a data model (as stated above). You can "up lift" or map data from XML to RDF (hierarchical to graph model mapping). Likewise you can "down shift" or map data from RDF to XML (example: SPARQL SELECT query patterns "down shift" to SPARQL Results XML, which isn't RDF/XML, while keeping access to graphs via URIs or Entity Identifiers that reside within the serialization).

Inaccuracy #5:

RDF extends the XML model and syntax to be specified for describing either resources or a collection of information. (XML points to a resource in order to scope and uniquely identify a set of properties known as the schema.).

Correction #5:

See earlier comments.

The single accurate paragraph in this ebiz article lies right at the end and it states the following:

"I've always thought RDF has been underutilized for data integration, and it's really an old standard. Now that we're focused on both understanding and integrating data, perhaps RDF should make a comeback."

Related:

• Semantic Web FAQ fragment re. RDF and XML
• Various posts re. RDF and Data Integration from this Blog Data Space.

What is the Data Access, and Data Management Value Pyramid?

As depicted below, a top-down view of the data access and data management value chain. The term: apex, simply indicates value primacy, which takes the form of a data access API based entry point into a DBMS realm -- aligned to an underlying data model. Examples of data access APIs include: Native Call Level Interfaces (CLIs), ODBC, JDBC, ADO.NET, OLE-DB, XMLA, and Web Services.

The degree to which ad-hoc views of data managed by a DBMS can be produced and dispatched to relevant data consumers (e.g. people), without compromising concurrency, data durability, and security, collectively determine the "Agility Value Factor" (AVF) of a given DBMS. Remember, agility as the cornerstone of environmental adaptation is as old as the concept of evolution, and intrinsic to all pursuits of primacy.

In simpler business oriented terms, look at AVF as the degree to which DBMS technology affects the ability to effectively implement "Market Leadership Discipline" along the following pathways: innovation, operation excellence, or customer intimacy.

Why has RDBMS Primacy has Endured?

Historically, at least since the late '80s, the RDBMS genre of DBMS has consistently offered the highest AVF relative to other DBMS genres en route to primacy within the value pyramid. The desire to improve on paper reports and spreadsheets is basically what DBMS technology has fundamentally addressed to date, even though conceptual level interaction with data has never been its forte.

For more then 10 years -- at the very least -- limitations of the traditional RDBMS in the realm of conceptual level interaction with data across diverse data sources and schemas (enterprise, Web, and Internet) has been crystal clear to many RDBMS technology practitioners, as indicated by some of the quotes excerpted below:

"Future of Database Research is excellent, but what is the future of data?"

"..it is hard for me to disagree with the conclusions in this report. It captures exactly the right thoughts, and should be a must read for everyone involved in the area of databases and database research in particular."

-- Dr. Anant Jingran, CTO, IBM Information Management Systems, commenting on the 2007 RDBMS technology retreat attended by a number of key DBMS technology pioneers and researchers.

"One size fits all: A concept whose time has come and gone

1. They are direct descendants of System R and Ingres and were architected more than 25 years ago
2. They are advocating "one size fits all"; i.e. a single engine that solves all DBMS needs.

-- Prof. Michael Stonebreaker, one of the founding fathers of the RDBMS industry.

Until this point in time, the requisite confluence of "circumstantial pain" and "open standards" based technology required to enable an objective "compare and contrast" of RDBMS engine virtues and viable alternatives hasn't occurred. Thus, the RDBMS has endured it position of primacy albeit on a "one size fits all basis".

Circumstantial Pain

As mentioned earlier, we are in the midst of an economic crisis that is ultimately about a consistent inability to connect dots across a substrate of interlinked data sources that transcend traditional data access boundaries with high doses of schematic heterogeneity. Ironically, in a era of the dot-com, we haven't been able to make meaningful connections between relevant "real-world things" that extend beyond primitive data hosted database tables and content management style document containers; we've struggled to achieve this in the most basic sense, let alone evolve our ability to connect inline with the exponential rate at which the Internet & Web are spawning "universes of discourse" (data spaces) that emanate from user activity (within the enterprise and across the Internet & Web). In a nutshell, we haven't been able to upgrade our interaction with data such that "conceptual models" and resulting "context lenses" (or facets) become concrete; by this I mean: real-world entity interaction making its way into the computer realm as opposed to the impedance we all suffer today when we transition from conceptual model interaction (real-world) to logical model interaction (when dealing with RDBMS based data access and data management).

Here are some simple examples of what I can only best describe as: "critical dots unconnected", resulting from an inability to interact with data conceptually:

Financial regulatory bodies couldn't effectively discern that a Credit Default Swap is an Insurance policy in all but literal name. And in not doing so the cost of an unregulated insurance policy laid the foundation for exacerbating the toxicity of fatally flawed mortgage backed securities. Put simply: a flawed insurance policy was the fallback on a toxic security that financiers found exotic based on superficial packaging.

Banks still don't understand that capital really does exists in tangible and intangible forms; with the intangible being the variant that is inherently dynamic. For example, a tech companies intellectual capital far exceeds the value of fixture, fittings, and buildings, but you be amazed to find that in most cases this vital asset has not significant value when banks get down to the nitty gritty of debt collateral; instead, a buffer of flawed securitization has occurred atop a borderline static asset class covering the aforementioned buildings, fixtures, and fittings.

In the general enterprise arena, IT executives continued to "rip and replace" existing technology without ever effectively addressing the timeless inability to connect data across disparate data silos generated by internal enterprise applications, let alone the broader need to mesh data from the inside with external data sources. No correlations made between the growth of buzzwords and the compounding nature of data integration challenges. It's 2009 and only a miniscule number of executives dare fantasize about being anywhere within distance of the: relevant information at your fingertips vision.

Looking more holistically at data interaction in general, whether you interact with data in the enterprise space (i.e., at work) or on the Internet or Web, you ultimately are delving into a mishmash of disparate computer systems, applications, service (Web or SOA), and databases (of the RDBMS variety in a majority of cases) associated with a plethora of disparate schemas. Yes, but even today "rip and replace" is still the norm pushed by most vendors; pitting one mono culture against another as exemplified by irrelevances such as: FOSS/LAMP vs Commercial or Web vs. Enterprise, when none of this matters if the data access and integration issues are recognized let alone addressed (see: Applications are Like Fish and Data Like Wine).

Like the current credit-crunch, exponential growth of data originating from disparate application databases and associated schemas, within shrinking processing time frames, has triggered a rethinking of what defines data access and data management value today en route to an inevitable RDBMS downgrade within the value pyramid.

Technology

There have been many attempts to address real-world modeling requirements across the broader DBMS community from Object Databases to Object-Relational Databases, and more recently the emergence of simple Entity-Attribute-Value model DBMS engines. In all cases failure has come down to the existence of one or more of the following deficiencies, across each potential alternative:

1. Query language standardization - nothing close to SQL standardization
2. Data Access API standardization - nothing close to ODBC, JDBC, OLE-DB, or ADO.NET
3. Wire protocol standardization - nothing close to HTTP
4. Distributed Identity infrastructure - nothing close to the non-repudiatable digital Identity that foaf+ssl accords
5. Use of Identifiers as network based pointers to data sources - nothing close to RDF based Linked Data
6. Negotiable data representation - nothing close to Mime and HTTP based Content Negotiation
7. Scalability especially in the era of Internet & Web scale.

Entity-Attribute-Value with Classes & Relationships (EAV/CR) data models

A common characteristic shared by all post-relational DBMS management systems (from Object Relational to pure Object) is an orientation towards variations of EAV/CR based data models. Unfortunately, all efforts in the EAV/CR realm have typically suffered from at least one of the deficiencies listed above. In addition, the same "one DBMS model fits all" approach that lies at the heart of the RDBMS downgrade also exists in the EAV/CR realm.

What Comes Next?

The RDBMS is not going away (ever), but its era of primacy -- by virtue of its placement at the apex of the data access and data management value pyramid -- is over! I make this bold claim for the following reasons:

1. The Internet aided "Global Village" has brought "Open World" vs "Closed World" assumption issues to the fore e.g., the current global economic crisis remains centered on the inability to connect dots across "Open World" and "Closed World" data frontiers
2. Entity-Attribute-Value with Classes & Relationships (EAV/CR) based DBMS models are more effective when dealing with disparate data associated with disparate schemas, across disparate DBMS engines, host operating systems, and networks.

Based on the above, it is crystal clear that a different kind of DBMS -- one with higher AVF relative to the RDBMS -- needs to sit atop today's data access and data management value pyramid. The characteristics of this DBMS must include the following:

1. Every item of data (Datum/Entity/Object/Resource) has Identity
2. Identity is achieved via Identifiers that aren't locked at the DBMS, OS, Network, or Application levels
3. Object Identifiers and Object values are independent (extricably linked by association)
4. Object values should be de-referencable via Object Identifier
5. Representation of de-referenced value graph (entity, attributes, and values mesh) must be negotiable (i.e. content negotiation)
6. Structured query language must provide mechanism for Creation, Deletion, Updates, and Querying of data objects
7. Performance & Scalability across "Closed World" (enterprise) and "Open World" (Internet & Web) realms.

Quick recap, I am not saying that RDBMS engine technology is dead or obsolete. I am simply stating that the era of RDBMS primacy within the data access and data management value pyramid is over.

The problem domain (conceptual model views over heterogeneous data sources) at the apex of the aforementioned pyramid has simply evolved beyond the natural capabilities of the RDBMS which is rooted in "Closed World" assumptions re., data definition, access, and management. The need to maintain domain based conceptual interaction with data is now palpable at every echelon within our "Global Village" - Internet, Web, Enterprise, Government etc.

It is my personal view that an EAV/CR model based DBMS, with support for the seven items enumerated above, can trigger the long anticipated RDBMS downgrade. Such a DBMS would be inherently multi-model because you would need to the best of RDBMS and EAV/CR model engines in a single product, with in-built support for HTTP and other Internet protocols in order to effectively address data representation and serialization issues.

EAV/CR Oriented Data Access & Management Technology

Examples of contemporary EAV/CR frameworks that provide concrete conceptual layers for data access and data management currently include:

• Resource Description Framework (RDF) - an EAV/CR based framework
• RDF Linked Data - EAV/CR based framework that mandates de-referencable HTTP based Identifiers
• ADO.NET Entity Frameworks - Microsoft .NET based EAV/CR framework
• Core Data Services - Mac OS X based EAV/CR framework that evolved from NeXT's Enterprise Object Frameworks (EOF).

The frameworks above provide the basis for a revised AVF pyramid, as depicted below, that reflects today's data access and management realities i.e., an Internet & Web driven global village comprised of interlinked distributed data objects, compatible with "Open World" assumptions.

Related

• How & Why Glue is Using Amazon SimpleDB
• Object Database Manifesto (Identity excerpt)
• Database Models Overview
• Ted Nelson Explaining Irregularity and Idiosyncrasy of Data Structures - ZigZag Demo

What is the Data Access, and Data Management Value Pyramid?

As depicted below, a top-down view of the data access and data management value chain. The term: apex, simply indicates value primacy, which takes the form of a data access API based entry point into a DBMS realm -- aligned to an underlying data model. Examples of data access APIs include: Native Call Level Interfaces (CLIs), ODBC, JDBC, ADO.NET, OLE-DB, XMLA, and Web Services.

The degree to which ad-hoc views of data managed by a DBMS can be produced and dispatched to relevant data consumers (e.g. people), without compromising concurrency, data durability, and security, collectively determine the "Agility Value Factor" (AVF) of a given DBMS. Remember, agility as the cornerstone of environmental adaptation is as old as the concept of evolution, and intrinsic to all pursuits of primacy.

In simpler business oriented terms, look at AVF as the degree to which DBMS technology affects the ability to effectively implement "Market Leadership Discipline" along the following pathways: innovation, operation excellence, or customer intimacy.

Why has RDBMS Primacy has Endured?

Historically, at least since the late '80s, the RDBMS genre of DBMS has consistently offered the highest AVF relative to other DBMS genres en route to primacy within the value pyramid. The desire to improve on paper reports and spreadsheets is basically what DBMS technology has fundamentally addressed to date, even though conceptual level interaction with data has never been its forte.

For more then 10 years -- at the very least -- limitations of the traditional RDBMS in the realm of conceptual level interaction with data across diverse data sources and schemas (enterprise, Web, and Internet) has been crystal clear to many RDBMS technology practitioners, as indicated by some of the quotes excerpted below:

"Future of Database Research is excellent, but what is the future of data?"

"..it is hard for me to disagree with the conclusions in this report. It captures exactly the right thoughts, and should be a must read for everyone involved in the area of databases and database research in particular."

-- Dr. Anant Jingran, CTO, IBM Information Management Systems, commenting on the 2007 RDBMS technology retreat attended by a number of key DBMS technology pioneers and researchers.

"One size fits all: A concept whose time has come and gone

1. They are direct descendants of System R and Ingres and were architected more than 25 years ago
2. They are advocating "one size fits all"; i.e. a single engine that solves all DBMS needs.

-- Prof. Michael Stonebreaker, one of the founding fathers of the RDBMS industry.

Until this point in time, the requisite confluence of "circumstantial pain" and "open standards" based technology required to enable an objective "compare and contrast" of RDBMS engine virtues and viable alternatives hasn't occurred. Thus, the RDBMS has endured it position of primacy albeit on a "one size fits all basis".

Circumstantial Pain

As mentioned earlier, we are in the midst of an economic crisis that is ultimately about a consistent inability to connect dots across a substrate of interlinked data sources that transcend traditional data access boundaries with high doses of schematic heterogeneity. Ironically, in a era of the dot-com, we haven't been able to make meaningful connections between relevant "real-world things" that extend beyond primitive data hosted database tables and content management style document containers; we've struggled to achieve this in the most basic sense, let alone evolve our ability to connect inline with the exponential rate at which the Internet & Web are spawning "universes of discourse" (data spaces) that emanate from user activity (within the enterprise and across the Internet & Web). In a nutshell, we haven't been able to upgrade our interaction with data such that "conceptual models" and resulting "context lenses" (or facets) become concrete; by this I mean: real-world entity interaction making its way into the computer realm as opposed to the impedance we all suffer today when we transition from conceptual model interaction (real-world) to logical model interaction (when dealing with RDBMS based data access and data management).

Here are some simple examples of what I can only best describe as: "critical dots unconnected", resulting from an inability to interact with data conceptually:

Financial regulatory bodies couldn't effectively discern that a Credit Default Swap is an Insurance policy in all but literal name. And in not doing so the cost of an unregulated insurance policy laid the foundation for exacerbating the toxicity of fatally flawed mortgage backed securities. Put simply: a flawed insurance policy was the fallback on a toxic security that financiers found exotic based on superficial packaging.

Banks still don't understand that capital really does exists in tangible and intangible forms; with the intangible being the variant that is inherently dynamic. For example, a tech companies intellectual capital far exceeds the value of fixture, fittings, and buildings, but you be amazed to find that in most cases this vital asset has not significant value when banks get down to the nitty gritty of debt collateral; instead, a buffer of flawed securitization has occurred atop a borderline static asset class covering the aforementioned buildings, fixtures, and fittings.

In the general enterprise arena, IT executives continued to "rip and replace" existing technology without ever effectively addressing the timeless inability to connect data across disparate data silos generated by internal enterprise applications, let alone the broader need to mesh data from the inside with external data sources. No correlations made between the growth of buzzwords and the compounding nature of data integration challenges. It's 2009 and only a miniscule number of executives dare fantasize about being anywhere within distance of the: relevant information at your fingertips vision.

Looking more holistically at data interaction in general, whether you interact with data in the enterprise space (i.e., at work) or on the Internet or Web, you ultimately are delving into a mishmash of disparate computer systems, applications, service (Web or SOA), and databases (of the RDBMS variety in a majority of cases) associated with a plethora of disparate schemas. Yes, but even today "rip and replace" is still the norm pushed by most vendors; pitting one mono culture against another as exemplified by irrelevances such as: FOSS/LAMP vs Commercial or Web vs. Enterprise, when none of this matters if the data access and integration issues are recognized let alone addressed (see: Applications are Like Fish and Data Like Wine).

Like the current credit-crunch, exponential growth of data originating from disparate application databases and associated schemas, within shrinking processing time frames, has triggered a rethinking of what defines data access and data management value today en route to an inevitable RDBMS downgrade within the value pyramid.

Technology

There have been many attempts to address real-world modeling requirements across the broader DBMS community from Object Databases to Object-Relational Databases, and more recently the emergence of simple Entity-Attribute-Value model DBMS engines. In all cases failure has come down to the existence of one or more of the following deficiencies, across each potential alternative:

1. Query language standardization - nothing close to SQL standardization
2. Data Access API standardization - nothing close to ODBC, JDBC, OLE-DB, or ADO.NET
3. Wire protocol standardization - nothing close to HTTP
4. Distributed Identity infrastructure - nothing close to the non-repudiatable digital Identity that foaf+ssl accords
5. Use of Identifiers as network based pointers to data sources - nothing close to RDF based Linked Data
6. Negotiable data representation - nothing close to Mime and HTTP based Content Negotiation
7. Scalability especially in the era of Internet & Web scale.

Entity-Attribute-Value with Classes & Relationships (EAV/CR) data models

A common characteristic shared by all post-relational DBMS management systems (from Object Relational to pure Object) is an orientation towards variations of EAV/CR based data models. Unfortunately, all efforts in the EAV/CR realm have typically suffered from at least one of the deficiencies listed above. In addition, the same "one DBMS model fits all" approach that lies at the heart of the RDBMS downgrade also exists in the EAV/CR realm.

What Comes Next?

The RDBMS is not going away (ever), but its era of primacy -- by virtue of its placement at the apex of the data access and data management value pyramid -- is over! I make this bold claim for the following reasons:

1. The Internet aided "Global Village" has brought "Open World" vs "Closed World" assumption issues to the fore e.g., the current global economic crisis remains centered on the inability to connect dots across "Open World" and "Closed World" data frontiers
2. Entity-Attribute-Value with Classes & Relationships (EAV/CR) based DBMS models are more effective when dealing with disparate data associated with disparate schemas, across disparate DBMS engines, host operating systems, and networks.

Based on the above, it is crystal clear that a different kind of DBMS -- one with higher AVF relative to the RDBMS -- needs to sit atop today's data access and data management value pyramid. The characteristics of this DBMS must include the following:

1. Every item of data (Datum/Entity/Object/Resource) has Identity
2. Identity is achieved via Identifiers that aren't locked at the DBMS, OS, Network, or Application levels
3. Object Identifiers and Object values are independent (extricably linked by association)
4. Object values should be de-referencable via Object Identifier
5. Representation of de-referenced value graph (entity, attributes, and values mesh) must be negotiable (i.e. content negotiation)
6. Structured query language must provide mechanism for Creation, Deletion, Updates, and Querying of data objects
7. Performance & Scalability across "Closed World" (enterprise) and "Open World" (Internet & Web) realms.

Quick recap, I am not saying that RDBMS engine technology is dead or obsolete. I am simply stating that the era of RDBMS primacy within the data access and data management value pyramid is over.

The problem domain (conceptual model views over heterogeneous data sources) at the apex of the aforementioned pyramid has simply evolved beyond the natural capabilities of the RDBMS which is rooted in "Closed World" assumptions re., data definition, access, and management. The need to maintain domain based conceptual interaction with data is now palpable at every echelon within our "Global Village" - Internet, Web, Enterprise, Government etc.

It is my personal view that an EAV/CR model based DBMS, with support for the seven items enumerated above, can trigger the long anticipated RDBMS downgrade. Such a DBMS would be inherently multi-model because you would need to the best of RDBMS and EAV/CR model engines in a single product, with in-built support for HTTP and other Internet protocols in order to effectively address data representation and serialization issues.

EAV/CR Oriented Data Access & Management Technology

Examples of contemporary EAV/CR frameworks that provide concrete conceptual layers for data access and data management currently include:

• Resource Description Framework (RDF) - an EAV/CR based framework
• RDF Linked Data - EAV/CR based framework that mandates de-referencable HTTP based Identifiers
• ADO.NET Entity Frameworks - Microsoft .NET based EAV/CR framework
• Core Data Services - Mac OS X based EAV/CR framework that evolved from NeXT's Enterprise Object Frameworks (EOF).

The frameworks above provide the basis for a revised AVF pyramid, as depicted below, that reflects today's data access and management realities i.e., an Internet & Web driven global village comprised of interlinked distributed data objects, compatible with "Open World" assumptions.

Related

• The Semantic Way - Alan Cho's Summary of PwC 2009 tech forecast report on the Semantic Web
• Is the RDBMS Doomed - ReadWriteWeb Article
• Anti-RDBMS: a list of Distributed Key-Value Stores - by Richard Jones (CTO Last.FM)
• How & Why Glue is Using Amazon SimpleDB
• Object Database Manifesto (Identity excerpt)
• Database Models Overview
• Ted Nelson Explaining Irregularity and Idiosyncrasy of Data Structures - ZigZag Demo

Simple example / demo:

Enter search pattern: Microsoft

You will get the usual result from a full text pattern search i.e., hits and text excerpts with matching patterns in boldface. This first step is akin to throwing your net out to sea while fishing.

Now you have your catch, what next? Basically, this is where traditional text search value ends since regex or xpath/xquery offer little when the structure of literal text is the key to filtering or categorization based analysis of real-world entities. Naturally, this is where the value of structured querying of linked data starts, as you seek to use entity descriptions (combination of attribute and relationship properties) to "Find relevant things".

Continuing with the demo.

Click on "Properties" link within the Navigation section of the browser page which results in a distillation and aggregation of the properties of the entities associated with the search results. Then use the "Next" link to page through the properties until to find the properties that best match what you seek. Note, this particular step is akin to using the properties of the catch (using fishing analogy) for query filtering, with each subsequent property link click narrowing your selection further.

Using property based filtering is just one perspective on the data corpus associated with the text search pattern; thus, you can alter perspectives by clicking on the "Class" link so that you can filter you search results by entity type. Of course, in a number of scenarios you would use a combination of entity types and entity properties filters to locate the entities of interest to you.

A Few Notes about this demo instance of Virtuoso:

• Lookup Data Size (Local Linked Data Corpus): 2 Billion+ Triples (entity-attribute-value tuples)
• This is a *temporary* teaser / precursor to the LOD (Linking Open Data Cloud) variant of our Linked Data driven "Search" & "Find" service; we decided to implement this functionality prior to commissioning a larger and more up to date instance based on the entire LOD Cloud
• The browser is simply using a Virtuoso PL function that also exists in Web Service form for loose binding by 3rd parties that have a UI orientation and focus (our UI is deliberately bare boned).
• The properties and entity types (classes) links expose formal definitions and dictionary provenance information materialized in an HTML page (of course your browser or any other HTTP user agent can negotiation alternative representations of this descriptive information)
• UMBEL based inference rules are enabled, giving you a live and simple demonstration of the virtues of Linked Data Dictionaries for example: click on the description link of any property or class from the foaf (friend-of-a-friend vocabulary), sioc (semantically-interlinked-online-communities ontology), mo (music ontology), bibo (bibliographic data ontology) namespaces to see how the data between these lower level vocabularies or ontologies are meshed with OpenCyc's upper level ontology.

Related

• Faceted Search: Unlimited Data in Interactive Time
• Virtuoso Anytime: No Query Is Too Complex

What is it?

A data access driver/provider that provides conceptual entity oriented access to RDBMS data managed by Virtuoso. Naturally, it also uses Virtuoso's in-built virtual / federated database layer to provide access to ODBC and JDBC accessible RDBMS engines such as: Oracle (7.x to latest), SQL Server (4.2 to latest), Sybase, IBM Informix (5.x to latest), IBM DB2, Ingres (6.x to latest), Progress (7.x to OpenEdge), MySQL, PostgreSQL, Firebird, and others using our ODBC or JDBC bridge drivers.

Benefits?

Technical:

It delivers an Entity-Attribute-Value + Classes & Relationships model over disparate data sources that are materialized as .NET Entity Framework Objects, which are then consumable via ADO.NET Data Object Services, LINQ for Entities, and other ADO.NET data consumers.

The provider is fully integrated into Visual Studio 2008 and delivers the same "ease of use" offered by Microsoft's own SQL Server provider, but across Virtuoso, Oracle, Sybase, DB2, Informix, Ingres, Progress (OpenEdge), MySQL, PostgreSQL, Firebird, and others. The same benefits also apply uniformly to Entity Frameworks compatibility.

Bearing in mind that Virtuoso is a multi-model (hybrid) data manager, this also implies that you can use .NET Entity Frameworks against all data managed by Virtuoso. Remember, Virtuoso's SQL channel is a conduit to Virtuoso's core; thus, RDF (courtesy of SPASQL as already implemented re. Jena/Sesame/Redland providers), XML, and other data forms stored in Virtuoso also become accessible via .NET's Entity Frameworks.

Strategic:

You can choose which entity oriented data access model works best for you: RDF Linked Data & SPARQL or .NET Entity Frameworks & Entity SQL. Either way, Virtuoso delivers a commercial grade, high-performance, secure, and scalable solution.

How do I use it?

• Using Visual Studio 2008 & Virtuoso to build an Entity Frameworks based Windows forms application
• Using Visual Studio 2008 & Virtuoso to build an ADO.NET Data Services based application

Note: When working with external or 3rd party databases, simply use the Virtuoso Conductor to link the external data source into Virtuoso. Once linked, the remote tables will simply be treated as though they are native Virtuoso tables leaving the virtual database engine to handle the rest. This is similar to the role the Microsoft JET engine played in the early days of ODBC, so if you've ever linked an ODBC data source into Microsoft Access, you are ready to do the same using Virtuoso.

Related

• Entity Oriented Data Access
• Yoda & the Data FORCE.

In 2009 I hope the following happens re. "Linked Data":

1. We realize it's a Meme
2. We collectively connect the Meme to the concept of granular hyperlinks between data entities/objects (datum to datum linkage aka. Hyperdata Linking)
3. We generally connect the Meme to technology ancestry such as the Entity-Attribute-Value with Classes & Relationships (EAV/CR) data model (then broader commonality with erstwhile unrelated realms will be unveiled e.g., Entity Frameworks from Microsoft, Core Data from Apple, SimpleDB from Amazon, and the Freebase Graph Model DB amongst others)
4. We instinctively connect the Meme to the concept of Entity Oriented Data Access and Management (RDF based Linked Data is basically EAV/CR scheme that uses HTTP based Pointers for Entity, Attribute, and Relationship Identifiers)
5. We naturally connect the Meme with the notion that an identifier for a unit of data (aka. Datum) should be the conduit to a negotiable representation of said Datum's description (i.e., it's attribute and relationship properties in HTML, XHTML, RDFa, Turtle, N3, RDF/XML etc., for example)
6. We ultimately connect the Meme with a conceptual-level approach to data integration across disparate data sources (also known as Master Data Management (MDM) ).

2009 is about a reboot on a monumental scale. We need new thinking, new technology, new approaches, and new solutions. No matter what route we take, we can't negate the importance of "Data". When dealing with organic or inorganic computers systems -- Data is simply everything!

The ability of individuals and enterprises to access, mesh, and disseminate data to relevant nodes across public and private networks will ultimately determine the winners and losers in the new frontier, ushered in by 2009.

Do not take data access and data management technology for granted. User interfaces come and ago, application logic comes and goes, but your data stays with you forever. If you are mystified by data access technology then make 2009 the year of data access technology demystification :-)

Related

• Linked Data & The Year 2009
• Various posts from my blog space

A pre-installed edition of Virtuoso for Amazon's EC2 Cloud platform.

What does it offer?

1. Low cost entry point to a game-changing Web 3.0+ (and beyond) platform that combines SQL, RDF, XML, and Web Services functionality
2. Flexible variable cost model (courtesy of EC2 DevPay) tightly bound to revenue generated by your services
3. Delivers federated and/or centralized model flexibility for you SaaS based solutions
4. Simple entry point for developing and deploying sophisticated database driven applications (SQL or RDF Linked Data Web oriented)
5. Complete framework for exploiting OpenID, OAuth (including Role enhancements) that simplifies exploitation of these vital Identity and Data Access technologies
6. Easily implement RDF Linked Data based Mail, Blogging, Wikis, Bookmarks, Calendaring, Discussion Forums, Tagging, Social-Networking as Data Space (data containers) features of your application or service offering
7. Instant alleviation of challenges (e.g. service costs and agility) associated with Data Portability and Open Data Access across Web 2.0 data silos
8. LDAP integration for Intranet / Extranet style applications.

From the DBMS engine perspective it provides you with one or more pre-configured instances of Virtuoso that enable immediate exploitation of the following services:

1. RDF Database (a Quad Store with SPARQL & SPARUL Language & Protocol support)
2. SQL Database (with ODBC, JDBC, OLE-DB, ADO.NET, and XMLA driver access)
3. XML Database (XML Schema, XQuery/Xpath, XSLT, Full Text Indexing)
4. Full Text Indexing.

From a Middleware perspective it provides:

1. RDF Views (Wrappers / Semantic Covers) over SQL, XML, and other data sources accessible via SOAP or REST style Web Services
2. Sponger Service for converting non RDF information resources into RDF Linked Data "on the fly" via a large collection of pre-installed RDFizer Cartridges.

From the Web Server Platform perspective it provides an alternative to LAMP stack components such as MySQL and Apace by offering

1. HTTP Web Server
2. WebDAV Server
3. Web Application Server (includes PHP runtime hosting)
4. SOAP or REST style Web Services Deployment
5. RDF Linked Data Deployment
6. SPARQL (SPARQL Query Language) and SPARUL (SPARQL Update Language) endpoints
7. Virtuoso Hosted PHP packages for MediaWiki, Drupal, Wordpress, and phpBB3 (just install the relevant Virtuoso Distro. Package).

From the general System Administrator's perspective it provides:

1. Online Backups (Backup Set dispatched to S3 buckets, FTP, or HTTP/WebDAV server locations)
2. Synchronized Incremental Backups to Backup Set locations
3. Backup Restore from Backup Set location (without exiting to EC2 shell).

Higher level user oriented offerings include:

1. OpenLink Data Explorer front-end for exploring the burgeoning Linked Data Web
2. Ajax based SPARQL Query Builder (iSPARQL) that enables SPARQL Query construction by Example
3. Ajax based SQL Query Builder (QBE) that enables SQL Query construction by Example.

For Web 2.0 / 3.0 users, developers, and entrepreneurs it offers it includes Distributed Collaboration Tools & Social Media realm functionality courtesy of ODS that includes:

1. Point of presence on the Linked Data Web that meshes your Identity and your Data via URIs
2. System generated Social Network Profile & Contact Data via FOAF?
3. System generated SIOC (Semantically Interconnected Online Community) Data Space (that includes a Social Graph) exposing all your Web data in RDF Linked Data form
4. System generated OpenID and automatic integration with FOAF
5. Transparent Data Integration across Facebook, Digg, LinkedIn, FriendFeed, Twitter, and any other Web 2.0 data space equipped with RSS / Atom support and/or REST style Web Services
6. In-built support for SyncML which enables data synchronization with Mobile Phones.

How Do I Get Going with It?

• Standard Installation Guide
• Personal or Service Specific DBpedia Installation Guide

Here is how I see Linked Data providing tangible value to MDM tools vendors and users:

1. Open access to Entities across MDM instances served up by different MDM solutions acting as Linked Data publishers (i.e., expose MDM Entities as RDF resources endowed with de-referencable URIs thereby enabling Hyperdata-style linking)
2. Use of RDF-ization middleware to hook disparate data sources (SQL, XML, and other data sources) into existing MDM packages (i.e., the MDM solutions become consumers of RDF Linked Data).

Of course Virtuoso was designed and developed to deliver the above from day one (circa. 1998 re. the core and 2005 re. the use of RDF for the final mile) as depicted below:

Related

• Other MDM related posts

Our diagram depicts the myriad of data sources from which RDF Linked Data is generated "on the fly" via our data source specific RDF-zation cartridges/drivers. It also unveils how the sponger leverages the Linked Data constellations of UMBEL, DBpedia, Bio2Rdf, and others for lookups.

From the RWW Top-Down category, which I interpret as: technologies that produce RDF from non RDF data sources. Our product portfolio is comprised of the following; Virtuoso Universal Server, OpenLink Data Spaces, OpenLink Ajax Toolkit, and OpenLink Data Explorer (which includes ubiquity commands).

Virtuoso Universal Server functionality summary:

1. Generation of RDF Linked Data Views of SQL, XML, and Web Services in general
2. Deployment of RDF Linked Data
3. "On the Fly" generation of RDF Linked Data from Document Web information resources (i.e. distillation of entities from their containers e.g. Web pages) via Cartridges / Drivers
4. SPARQL query language support
5. SPARQL extensions that bring SPARQL closer to SQL e.g Aggregates, Update, Insert, Delete Named Graph support (i.e. use of logical names to partition RDF data within Virtuoso's multi-model dbms engine)
6. Inference Engine (currently in use re. DBpedia via Yago and UMBEL)
7. Host and exposes data from Drupal, Wordpress, MediaWiki, phpBB3 as RDF Linked Data via in-built support for PHP runtime
8. Available as an EC2 AMI
9. etc..

OpenLink Data Spaces functionality summary:

1. Simple mechanism for Linked Data Web enabling yourself by giving you an HTTP based User ID (a de-referencable URI) that is linked to a FOAF based Profile page and OpenID
2. Binds all your data sources (blogs, wikis, bookmarks, photos, calendar items etc. ) to your URI so can "Find" things by only remembering your URI
3. Makes your profile page and personal URI the focal point of Linked Data Web presence
4. Delivers Data Portability (using data access by value or data access by reference) across data silos (e.g. Web 2.0 style social networks)
5. Allows you make annotations about anything in your own Data Space(s) on the Web without exposure to RDF markup
6. A Briefcase feature that provides a WebDAV driven RDF Linked Data variant of functionality seen in Mac OS X Spotlight and WinFS with the addition of SPARQL compliance
7. Automatically generates RDFa in its (X)HTML pages
8. Blog, Wiki, WebDAV File Server, Shared Bookmarks, Calendar, and other applications that look and feel like Web 2.0 counterparts but emitt RDF Linked Data amongst a plethora of data exchange formats
9. Available as an EC2 AMI
10. etc..

OpenLink Ajax Toolkit functionality summary:

1. Provides binding to SQL, RDF, XML, and Web Services via Ajax Database Connectivity Layer (you only need an ODBC, JDBC, OLE-DB, ADO.NET, XMLA Driver, or Web Service on the backend for dynamic data access from Javascript)
2. All controls are Ajax Database Connectivity bound (widgets get their data from Ajax Database Connectivity data sources)
3. Bundled with Virtuoso and ODS installations.
4. etc.

OpenLink Data Explorer functionality summary

1. Distills entities associated with information resource style containers (e.g. Web Pages or files) as RDF Linked Data
2. Exposes the RDF based Linked Data graph associated with information resources (see the Linked Data behind Web pages)
3. Ubiquity commands for invoking the above
4. Available as a Hosted Service or Firefox Extension
5. Bundled with Virtuoso and ODS installations
6. etc.

Note:

Of course you could have simply looked up OpenLink Software's FOAF based Profile page (*note the Linked Data Explorer tab*), or simply passed the FOAF profile page URL to a Linked Data aware client application such as: OpenLink Data Explorer, Zitgist Data Viewer, Marbles, and Tabulator, and obtained information. Remember, OpenLink Software is an Entity of Type: foaf:Organization, on the burgeoning Linked Data Web :-)

Related

• Linked Data Planet Keynote (RDFa based remix edition)
• On The Cusp: A Global Review of the Semantic Web Industry.

Understanding potential Linked Data Web business models, relative to other Web based market segments, is best pursued via a BCG Matrix diagram, such as the one I've constructed below:

Notes:

Link Density

• Web 1.0's collection of "Web Sites" have relatively low link density relative to Web 2.0's user-activity driven generation of semi-structured linked data spaces (e.g., Blogs, Wikis, Shared Bookmarks, RSS/Atom Feeds, Photo Galleries, Discussion Forums etc..)
• Semantic Technologies (i.e. "Semantics Inside style solutions") which are primarily about "Semantic Meaning" culled from Web 1.0 Pages also have limited linked density relative to Web 2.0
• The Linked Data Web, courtesy of the open-ended linking capacity of URIs, matches and ultimately exceeds Web 2.0 link density.

Relevance

• Web 1.0 and 2.0 are low relevance realms driven by hyperlinks to information resources ((X)HTML, RSS, Atom, OPML, XML, Images, Audio files etc.) associated with Literal Labels and Tagging schemes devoid of explicit property based resource description thereby making the pursuit of relevance mercurial at best
• Semantic Technologies offer more relevance than Web 1.0 and 2.0 based on the increased context that semantic analysis of Web pages accords
• The Linked Data Web, courtesy of URIs that expose self-describing data entities, match the relevance levels attained by Semantic Technologies.

Serendipity Quotient (SDQ)

• Web 1.0 has next to no serendipity, the closest thing is Google's "I'm Feeling Lucky" button
• Web 2.0 possess higher potential for serendipitous discovery than Web 1.0, but such potential is neutralized by inherent subjectivity due to its human-interaction-focused literal foundation (e.g., tags, voting schemes, wiki editors etc.)
• Semantic Technologies produce islands-of-relevance with little scope for serendipitous discovery due to URI invisibility, since the prime focus is delivering more context to Web search relative to traditional Web 1.0 search engines.
• The Linked Data Web's use of URIs as the naming and resolution mechanism for exposing structured and interlinked resources provides the highest potential for serendipitous discovery of relevant "Things"

To conclude, the Linked Data Web's market opportunities are all about the evolution of the Web into a powerful substrate that offers a unique intersection of "Link Density" and "Relevance", exploitable across horizontal and vertical market segments to solutions providers. Put differently, SDQ is how you take "The Ad" out of "Advertising" when matching Web users to relevant things :-)

CrunchBase: When we released the CrunchBase API, you were one of the first developers to step up and quickly released a CrunchBase Sponger Cartridge. Can you explain what a CrunchBase Sponger Cartridge is?

Me: A Sponger Cartridge is a data access driver for Web Resources that plugs into our Virtuoso Universal Server (DBMS and Linked Data Web Server combo amongst other things). It uses the internal structure of a resource and/or a web service associated with a resource, to materialize an RDF based Linked Data graph that essentially describes the resource via its properties (Attributes & Relationships).

CrunchBase: And what inspired you to create it?

Me: Bengee built a new space with your data, and we've built a space on the fly from your data which still resides in your domain. Either solution extols the virtues of Linked Data i.e. the ability to explore relationships across data items with high degrees of serendipity (also colloquially known as: following-your-nose pattern in Semantic Web circles).

Bengee posted a notice to the Linking Open Data Community's public mailing list announcing his effort. Bearing in mind the fact that we've been using middleware to mesh the realms of Web 2.0 and the Linked Data Web for a while, it was a no-brainer to knock something up based on the conceptual similarities between Wikicompany and CrunchBase. In a sense, a quadrant of orthogonality is what immediately came to mind re. Wikicompany, CrunchBase, Bengee's RDFization efforts, and ours.

Bengee created an RDF based Linked Data warehouse based on the data exposed by your API, which is exposed via the Semantic CrunchBase data space. In our case we've taken the "RDFization on the fly" approach which produces a transient Linked Data View of the CrunchBase data exposed by your APIs. Our approach is in line with our world view: all resources on the Web are data sources, and the Linked Data Web is about incorporating HTTP into the naming scheme of these data sources so that the conventional URL based hyperlinking mechanism can be used to access a structured description of a resource, which is then transmitted using a range negotiable representation formats. In addition, based on the fact that we house and publish a lot of Linked Data on the Web (e.g. DBpedia, PingTheSemanticWeb, and others), we've also automatically meshed Crunchbase data with related data in DBpedia and Wikicompany data.

CrunchBase: Do you know of any apps that are using CrunchBase Cartridge to enhance their functionality?

Me: Yes, the OpenLink Data Explorer which provides CrunchBase site visitors with the option to explore the Linked Data in the CrunchBase data space. It also allows them to "Mesh" (rather than "Mash") CrunchBase data with other Linked Data sources on the Web without writing a single line of code.

CrunchBase: You have been immersed in the Semantic Web movement for a while now. How did you first get interested in the Semantic Web?

Me: We saw the Semantic Web as a vehicle for standardizing conceptual views of heterogeneous data sources via context lenses (URIs). In 1998 as part of our strategy to expand our business beyond the development and deployment of ODBC, JDBC, and OLE-DB data providers, we decided to build a Virtual Database Engine (see: Virtuoso History), and in doing so we sought a standards based mechanism for the conceptual output of the data virtualization effort. As of the time of the seminal unveiling of the Semantic Web in 1998 we were clear about two things, in relation to the effects of the Web and Internet data management infrastructure inflections: 1) Existing DBMS technology had reached it limits 2) Web Servers would ultimately hit their functional limits. These fundamental realities compelled us to develop Virtuoso with an eye to leveraging the Semantic Web as a vehicle from completing its technical roadmap.

CrunchBase: Can you put into layman’s terms exactly what RDF and SPARQL are and why they are important? Do they only matter for developers or will they extend past developers at some point and be used by website visitors as well?

Me: RDF (Resource Description Framework) is a Graph based Data Model that facilitates resource description using the Subject, Predicate, and Object principle. Associated with the core data model, as part of the overall framework, are a number of markup languages for expressing your descriptions (just as you express presentation markup semantics in HTML or document structure semantics in XML) that include: RDFa (simple extension of HTML markup for embedding descriptions of things in a page), N3 (a human friendly markup for describing resources), RDF/XML (a machine friendly markup for describing resources).

SPARQL is the query language associated with the RDF Data Model, just as SQL is a query language associated with the Relational Database Model. Thus, when you have RDF based structured and linked data on the Web, you can query against Web using SPARQL just as you would against an Oracle/SQL Server/DB2/Informix/Ingres/MySQL/etc.. DBMS using SQL. That's it in a nutshell.

CrunchBase: On your website you wrote that “RDF and SPARQL as productivity boosters in everyday web development”. Can you elaborate on why you believe that to be true?

Me: I think the ability to discern a formal description of anything via its discrete properties is of immense value re. productivity, especially when the capability in question results in a graph of Linked Data that isn't confined to a specific host operating system, database engine, application or service, programming language, or development framework. RDF Linked Data is about infrastructure for the true materialization of the "Information at Your Fingertips" vision of yore. Even though it's taken the emergence of RDF Linked Data to make the aforementioned vision tractable, the comprehension of the vision's intrinsic value have been clear for a very long time. Most organizations and/or individuals are quite familiar with the adage: Knowledge is Power, well there isn't any knowledge without accessible Information, and there isn't any accessible Information without accessible Data. The Web has always be grounded in accessibility to data (albeit via compound container documents called Web Pages).

Bottom line, RDF based Linked Data is about Open Data access by reference using URIs (HTTP based Entity IDs / Data Object IDs / Data Source Names), and as I said earlier, the intrinsic value is pretty obvious bearing in mind the costs associated with integrating disparate and heterogeneous data sources -- across intranets, extranets, and the Internet.

CrunchBase: In his definition of Web 3.0, Nova Spivack proposes that the Semantic Web, or Semantic Web technologies, will be force behind much of the innovation that will occur during Web 3.0. Do you agree with Nova Spivack? What role, if any, do you feel the Semantic Web will play in Web 3.0?

Me: I agree with Nova. But I see Web 3.0 as a phase within the Semantic Web innovation continuum. Web 3.0 exists because Web 2.0 exists. Both of these Web versions express usage and technology focus patterns. Web 2.0 is about the use of Open Source technologies to fashion Web Services that are ultimately used to drive proprietary Software as Service (SaaS) style solutions. Web 3.0 is about the use of "Smart Data Access" to fashion a new generation of Linked Data aware Web Services and solutions that exploit the federated nature of the Web to maximum effect; proprietary branding will simply be conveyed via quality of data (cleanliness, context fidelity, and comprehension of privacy) exposed by URIs.

Here are some examples of the CrunchBase Linked Data Space, as projected via our CruncBase Sponger Cartridge:

1. Amazon.com
2. Microsoft
3. Google
4. Apple

By coincidence, Glenn and I presented at this month's Cambridge Semantic Web Gathering.

I've provided a dump of Glenn's issues and my responses below:

Issue - RDF

• Ingenious data decomposition idea, but:
• too low-level; the assembly language of data, where we need Java or Ruby
• "resource" is not the issue; there's no such thing as "metadata", it's all data; "meta" is a perspective
• lists need to be effortless, not painful and obscure
• nodes need to be represented, not just implied; they need types and literals in a more pervasive, integrated way.

Response:

RDF is a Graph based Data Model it stands for Resource Description Framework. The Metadata data angle comes from it's Meta Content Framework (MCF) origins. You can express and serialize data based on the RDF Data Model using: Turtle, N3, TriX, N-Triples, and RDF/XML.

Issue - SPARQL (and Freebase's MQL)

These are just appeasement:
- old query paradigm: fishing in dark water with superstitiously tied lures; only works well in carefully stocked lakes
- we don't ask questions by defining answer shapes and then hoping they're dredged up whole.

Response:

SPARQL, MQL, and Entity-SQL are Graph Model oriented Query Languages. Query Languages always accompany Database Engines. SQL is the Relational Model equivalent.

Issue - Linked Data

Noble attempt to ground the abstract, but:
- URI dereferencing/namespace/open-world issues focus too much technical attention on cross-source cases where the human issues dwarf the technical ones anyway
- FOAF query over the people in this room? forget it.
- link asymmetry doesn't scale
- identity doesn't scale
- generating RDF from non-graph sources: more appeasement, right where the win from actually converting could be biggest!

Response:

Innovative use of HTTP to deliver "Data Access by Reference" to the Linked Data Web.

When you have a Data Model, Database Engine, and Query Language, the next thing you need is a Data Access mechanism that provides "Data Access by Reference". ODBC and JDBC (amongst others) provide "Data Access by Reference" via Data Source Names. Linked Data is about the same thing (URIs are Data Source Names) with the following differences:

• Naming is scoped to the entity level rather than container level
• HTTP's use within the data source naming scheme expands the referencability of the Named Entity Descriptions beyond traditional confines such as applications, operating systems, and database engines.

Issue - Giant Global Graph

Hugely motivating and powerful idea, worthy of a superhero (Graphius!), but:
- giant and global parts are too hard, and starting global makes every problem harder
- local projects become unmanageable in global context (Cyc, Freebase data-modeling lists...). And my thus my plea, again. Forget "semantic" and "web", let's fix the database tech first:
- node/arc data-model, path-based exploratory query-model
- data-graph applications built easily on top of this common model; building them has to be easy, because if it's hard, they'll be bad
- given good database tech, good web data-publishing tech will be trivial!
- given good tools for graphs, the problems of uniting them will be only as hard as they have to be.

Response:

Giant Global Graph is just another moniker for a "Web of Linked Data" or "Linked Data Web".

Multi-Model Database technology that meshes the best of the Graph & Relational Models exist. In a nutshell, this is what Virtuoso is all about and it's existed for a very long time :-)

Virtuoso is also a Virtual DBMS engine (so you can see Heterogeneous Relational Data via Graph Model Context Lenses). Naturally, it is also a Linked Data Deployment platform (or Linked Data Sever).

The issue isn't the "Semantic Web" moniker per se., it's about how Linked Data (foundation layer of Semantic Web) gets introduced to users. As I said during the MIT Gathering: "The Web is experienced via Web Browsers primarily, so any enhancement to the Web must be exposed via traditional Web Browsers", which is why we've opted to simply add "View Linked Data Sources" to the existing set of common Browser options that includes:

1. View page in rendered form (default)
2. View page source (i.e., how you see the markup behind the page)

By exposing the Linked Data Web option as described above, you enable the Web user to knowingly transition from the traditional Rendered (X)HTML page view to the Linked Data View (i.e., structured data behind the page). This simple "User Interaction" tweak makes the notion of exploiting a Structured Web becomes somewhat clearer.

The Linked Data Web isn't a panacea. It's just an addition to the existing Web that enrichens the things you can do with the Web. It's predominance, like any application feature, will be subject to the degrees to which it delivers tangible value or matrializes internal and external opportunity costs.

Note: The Web isn't ubiquitous today becuase all it's users groked HTML Markup. It's ubquitity is a function of opportunity costs: there simply came a point in the Web boostrap when nobody could afford the opportunity costs associated with being off the Web. The same thing will play out with Linked Data and the broader Semantic Web vision.

1. Linked Data Journey part of my Linked Data Planet Presentation Remix(from slides 15 to 22 - which include bits from TimBL's presentation)
2. OpenLink Data Explorer
3. OpenLink Data Explorer Screenshots and examples.

The LinqToRdf project is about binding LINQ to RDF. It sits atop Joshua Tauberer's C# based Semantic Web/RDF library which has been out there for a while and works across Microsoft .NET and it's open source variant "Mono".

Historically, the Semantic Web realm has been dominated by RDF frameworks such as Sesame, Jena and Redland; which by their Open Source orientation, predominantly favor non-Windows platforms (Java and Linux). Conversely, Microsoft's .NET frameworks have sought to offer Conceptualization technology for heterogeneous Logical Data Sources via .NET's Entity Frameworks and ADO.NET, but without any actual bindings to RDF.

Interestingly, believe it or not, .NET already has a data query language that shares a number of similarities with SPARQL, called Entity-SQL, and a very innovative programming language called LINQ; that offers a blend of constructs for natural data access and manipulation across relational (SQL), hierarchical (XML), and graph (Object) models without the traditional object language->database impedance tensions of the past.

With regards to all of the above, we've just released a mini white paper that covers the exploitation of RDF-based Linked Data using .NET via LINQ. The paper offers a an overview of LinqToRdf, plus enhancements we've contributed to the project (available in LinqToRdf v0.8.). The paper includes real-world examples that tap into a MusicBrainz powered Linked Data Space, the Music Ontology, the Virtuoso RDF Quad Store, Virtuoso Sponger Middleware, and our RDfization Cartridges for Musicbrainz.

Here is the list:

1. Linked Data Deployment (Daniel Lewis, OpenLink Software)
2. The Programmes Ontology (Tom Scott, BBC and all)
3. SemWebbing the London Gazette (Jeni Tennison, The Stationery Office)
4. Searching, publishing and remixing a Web of Semantic Data (Richard Cyganiak, DERI Galway)
5. Building a Semantic Web Search Engine: Challenges and Solutions (Aidan Hogan, DERI Galway)
6. 'That's not what you said yesterday!' - evolving your Web API (Ian Davis, Talis)
7. Representing, indexing and mining scientific data using XML and RDF: Golem and CrystalEye (Andrew Walkingshaw, University of Cambridge)

For the time challenged (i.e. those unable to view this post using it's permalink / URI as a data source via the OpenLink RDF Browser, Zitgist Data Viewer, DISCO Hyperdata Browser, or Tabulator), the benefits of this post are as follows:

• automatic URI generation for all linked items in this post
• automatic propagation of tags to del.icio.us, Technorati, and PingTheSemanticWeb
• automatic association of formal meanings to my Tags using the MOAT Ontology
• automatic collation and generation of statistical data about my tags using the SCOT Ontology (*missing link is a callout to SCOT Tag Ontology folks to sort the project's home page URL at the very least*)
• explicit typing of my Tags as SKOS Concepts.

Put differently, I cost-effectively contribute to the GGG across all Web interaction dimensions (1.0, 2.0, 3.0) :-)

- Produce RDF Linked Data from non RDF Data Sources (Heterogeneous SQL, XML, Web Services)

- Provide highly scalable RDF Data Management via a Quad Store (DBpedia is an example of a live demonstration)

- Sophisticated Deployment of Linked Data that exploits the power of SPARQL

- Powerful WebDAV innovations that simplify read-write mode interaction with Linked Data

- Use Web Data Virtualization to address the pain and frustration associated with Web Data Silos (e.g. OpenLink Data Spaces layer stop Virtuoso that delivers Personal Data Spaces / Unified Storage in the Clouds)

- Deliver a Linked Data development and deployment platform to .NET (VB, C#) , Java, PHP, Ruby, Perl, Python, 'C', C++, and other developers

- More...

ReadWriteWeb via Alex Iskold's post have delivered another iteration of their "Guide to Semantic Technologies".

If you look at the title of this post (and their article) they seem to be accurately providing a guide to Semantic Technologies, so no qualms there. If on the other hand, this is supposed to he a guide to the "Semantic Web" as prescribed by TimBL then they are completely missing the essence of the whole subject, and demonstrably so I may add, since the entities: "ReadWriteWeb" and "Alex Iskold" are only describable today via the attributes of the documents they publish i.e their respective blogs and hosted blog posts.

Preoccupation with Literal objects as describe above, implies we can only take what "ReadWriteWeb" and "Alex Iskold" say "Literally" (grep, regex, and XPath/Xquery are the only tools for searching deeper in this Literal realm), we have no sense of what makes them tick or where they come from, no history (bar "About Page" blurb), no data connections beyond anchored text (more pointers to opaque data sources) in post and blogrolls. The only connection between this post and them is the my deliberate use of the same literal text in the Title of this post.

TimBL's vision as espoused via the "Semantic Web" vision is about the production, consumption, and sharing of Data Objects via HTTP based Identifiers called URIs/IRIs (Hyperdata Links / Linked Data). It's how we use the Web as a Distributed Database where (as Jim Hendler once stated with immense clarity): I can point to records (entity instances) in your database (aka Data Space) from mine. Which is to say that if we can all point to data entities/objects (not just data entities of type "Document") using these Location, Value, and Structure independent Object Identifiers (courtesy of HTTP) we end up with a much more powerful Web, and one that is closer to the "Federated and Open" nature of the Web.

As I stated in a prior post, if you or your platform of choice aren't producing de-referencable URIs for your data objects, you may be Semantic (this data model predates the Web), but there is no "World Wide Web" in what you are doing.

What are the Benefits of the Semantic Web?

Consumer - "Discovery of relevant things" and be being "Discovered by relevant things" (people, places, events, and other things)

Enterprise - ditto plus the addition of enterprise domain specific things such as market opportunities, product portfolios, human resources, partners, customers, competitors, co-opetitors, acquisition targets, new regulation etc..)

Simple demo:

I am a Kingsley Idehen, a Person who authors this weblog. I also share bookmarks gathered over the years across an array of subjects via my bookmark data space. I also subscribe to a number of RSS/Atom/RDF feeds, which I share via my feeds subscription data space. Of course, all of these data sources have Tags which are collectively exposed via my weblog tag-cloud, feeds subscriptions tag-cloud, and bookmarks tag-cloud data spaces.

As I don't like repeating myself, and I hate wasting my time or the time of others, I simply share my Data Space (a collection of all of my purpose specific data spaces) via the Web so that others (friends, family, employees, partners, customers, project collaborators, competitors, co-opetitors etc.) can can intentionally or serendipitously discover relevant data en route to creating new information (perspectives) that is hopefully exposed others via the Web.

Bottom-line, the Semantic Web is about adding the missing "Open Data Access & Connectivity" feature to the current Document Web (we have to beyond regex, grep, xpath, xquery, full text search, and other literal scrapping approaches). The Linked Data Web of de-referencable data object URIs is the critical foundation layer that makes this feasible.

Remember, It's not about "Applications" it's about Data and actually freeing Data from the "tyranny of Applications". Unfortunately, application inadvertently always create silos (esp. on the Web) since entity data modeling, open data access, and other database technology realm matters, remain of secondary interest to many application developers.

Final comment, RDF facilitates Linked Data on the Web, but all RDF isn't endowed with de-referencable URIs (a major source of confusion and misunderstanding). Thus, you can have RDF Data Source Providers that simply project RDF data silos via Web Services APIs if RDF output emanating from a Web Service doesn't provide out-bound pathways to other data via de-referencable URIs. Of course the same also applies to Widgets that present you with all the things they've discovered without exposing de-referencable URIs for each item.

BTW - my final comments above aren't in anyway incongruent with devising successful business models for the Web. As you may or may not know, OpenLink is not only a major platform provider for the Semantic Web (expressed in our UDA, Virtuoso, OpenLink Data Spaces, and OAT products), we are also actively seeding Semantic Web (tribe: Linked Data of course) startups. For instance, Zitgist, which now has Mike Bergman as it's CEO alongside Frederick Giasson as CTO. Of course, I cannot do Zitgist justice via a footnote in a blog post, so I will expand further in a separate post.

Additional information about this blog post:

1. I didn't spent hours looking for URIs used in my hyperlinks
2. The post is best viewed via an RDF Linked Data aware user agents (OpenLink RDF Browser, Zitgist Data Viewer, DISCO Hyperdata Browser, Tabulator).

If you look at the title of this post (and their article) they seem to be accurately providing a guide to Semantic Technologies, so no qualms there. If on the other hand, this is supposed to he a guide to the "Semantic Web" as prescribed by TimBL then they are completely missing the essence of the whole subject, and demonstrably so I may add, since the entities: "ReadWriteWeb" and "Alex Iskold" are only describable today via the attributes of the documents they publish i.e their respective blogs and hosted blog posts.

Preoccupation with Literal objects as describe above, implies we can only take what "ReadWriteWeb" and "Alex Iskold" say "Literally" (grep, regex, and XPath/Xquery are the only tools for searching deeper in this Literal realm), we have no sense of what makes them tick or where they come from, no history (bar "About Page" blurb), no data connections beyond anchored text (more pointers to opaque data sources) in post and blogrolls. The only connection between this post and them is the my deliberate use of the same literal text in the Title of this post.

TimBL's vision as espoused via the "Semantic Web" vision is about the production, consumption, and sharing of Data Objects via HTTP based Identifiers called URIs/IRIs (Hyperdata Links / Linked Data). It's how we use the Web as a Distributed Database where (as Jim Hendler once stated with immense clarity): I can point to records (entity instances) in your database (aka Data Space) from mine. Which is to say that if we can all point to data entities/objects (not just data entities of type "Document") using these Location, Value, and Structure independent Object Identifiers (courtesy of HTTP) we end up with a much more powerful Web, and one that is closer to the "Federated and Open" nature of the Web.

As I stated in a prior post, if you or your platform of choice aren't producing de-referencable URIs for your data objects, you may be Semantic (this data model predates the Web), but there is no "World Wide Web" in what you are doing.

What are the Benefits of the Semantic Web?

Consumer - "Discovery of relevant things" and be being "Discovered by relevant things" (people, places, events, and other things)

Enterprise - ditto plus the addition of enterprise domain specific things such as market opportunities, product portfolios, human resources, partners, customers, competitors, co-opetitors, acquisition targets, new regulation etc..)

Simple demo:

I am a Kingsley Idehen, a Person who authors this weblog. I also share bookmarks gathered over the years across an array of subjects via my bookmark data space. I also subscribe to a number of RSS/Atom/RDF feeds, which I share via my feeds subscription data space. Of course, all of these data sources have Tags which are collectively exposed via my weblog tag-cloud, feeds subscriptions tag-cloud, and bookmarks tag-cloud data spaces.

As I don't like repeating myself, and I hate wasting my time or the time of others, I simply share my Data Space (a collection of all of my purpose specific data spaces) via the Web so that others (friends, family, employees, partners, customers, project collaborators, competitors, co-opetitors etc.) can can intentionally or serendipitously discover relevant data en route to creating new information (perspectives) that is hopefully exposed others via the Web.

Bottom-line, the Semantic Web is about adding the missing "Open Data Access & Connectivity" feature to the current Document Web (we have to beyond regex, grep, xpath, xquery, full text search, and other literal scrapping approaches). The Linked Data Web of de-referencable data object URIs is the critical foundation layer that makes this feasible.

Remember, It's not about "Applications" it's about Data and actually freeing Data from the "tyranny of Applications". Unfortunately, application inadvertently always create silos (esp. on the Web) since entity data modeling, open data access, and other database technology realm matters, remain of secondary interest to many application developers.

Final comment, RDF facilitates Linked Data on the Web, but all RDF isn't endowed with de-referencable URIs (a major source of confusion and misunderstanding). Thus, you can have RDF Data Source Providers that simply project RDF data silos via Web Services APIs if RDF output emanating from a Web Service doesn't provide out-bound pathways to other data via de-referencable URIs. Of course the same also applies to Widgets that present you with all the things they've discovered without exposing de-referencable URIs for each item.

BTW - my final comments above aren't in anyway incongruent with devising successful business models for the Web. As you may or may not know, OpenLink is not only a major platform provider for the Semantic Web (expressed in our UDA, Virtuoso, OpenLink Data Spaces, and OAT products), we are also actively seeding Semantic Web (tribe: Linked Data of course) startups. For instance, Zitgist, which now has Mike Bergman as it's CEO alongside Frederick Giasson as CTO. Of course, I cannot do Zitgist justice via a footnote in a blog post, so I will expand further in a separate post.

Additional information about this blog post:

1. I didn't spent hours looking for URIs used in my hyperlinks
2. The post is best viewed via an RDF Linked Data aware user agents (OpenLink RDF Browser, Zitgist Data Viewer, DISCO Hyperdata Browser, Tabulator).

Yes, integration is hard, but I do profoundly believe that what's been happening on the Web over the last 10 or so years also applies to the Enterprise, and by this I absolutely do not mean "Enterprise 2.0" since "2.0" and productive agility do not compute in my realm of discourse.

large collections of RSS feeds, Wikiwords, Shared Bookmarks, Discussion Forums etc.. when disconnected at the data level (i.e. hosted in pages with no access to the "data behind") simply offer information deluge and inertia (there are only so many hours for processing opaque information sources in a given day).

Enterprises fundamentally need to process information efficiently as part of a perpetual assessment of their relative competitive Strengths, Weaknesses, Opportunities, and Threats (SWOT), in existing and/or future markets. Historically, IT acquisitions have run counter intuitively to the aforementioned quest for "Ability" due to the predominance of "rip and replace" approach technology acquisition that repeatedly creates and perpetuates information silos across Application, Database, Operating System, Development Environment boundaries. The sequence of events typically occurs as follows:

1. applications are acquired on a problem by problem basis
2. back-end application databases are discovered once ad-hoc information views are sought by information workers
3. back-end database disparity across applications is discovered once holistic views are sought by knowledge workers (typically domain experts).

In the early to mid 90's (pre ubiquitous Web), operating system, programming language, operating system, and development framework independence inside the enterprise was technically achievable via ODBC (due to it's platform independence). That said, DBMS specific ODBC channels alone couldn't address the holistic requirements associated with Conceptual Views of disparate data sources, hence the need for Data Access Virtualization via Virtual Database Engine technology.

Just as is the case on the Web today, with the emergence of the "Linked Data" meme, enterprises now have a powerful mechanism for exploiting the Data Integration benefits associated with generating Data Objects from disparate data sources, endowed with HTTP based IDs (URIs).

Conceptualizing access to data exposed Databases APIs, SOA based Web Services (SOAP style Web Services), Web 2.0 APIs (REST style Web Services), XML Views of SQL Data (SQLX), pure XML etc.. is problem area addressed by RDF aware middleware (RDFizers e.g Virtuoso Sponger).

Here are examples of what SQL Rows exposed as RDF Data Objects (identified using HTTP based URIs) would look like outside or behind a corporate firewall:

Customer - Alfreds Futterkiste

Customer Contact - Maria Anders

Salesrep - Nancy Davolio

Customer Orders Numbers - 11084, 11011, 11078, 11085

What's Good for the Web Goose (Personal Data Space URIs) is good for the Enterprise Gander (Enterprise Data Space URIs).

Related

Data Access - A Cultural or Technical Challenge?

*On* the ubiquitous Web of "Linked Documents", HREF means (by definition and usage): Hypertext Reference to an HTTP accessible Data Object of Type: "Document" (an information resource). Of course we don't make the formal connection of Object Type when dealing with the Web on a daily basis, but whenever you encounter the "resource not found" condition notice the message: HTTP/1.0 404 Object Not Found, from the HTTP Server tasked with retrieving and returning the resource.

*In* the Web of "Linked Data", a complimentary addition to the current Web of "Linked Documents", HREF is used to reference Data Objects that are of a variety of "Types", not just "Documents". And the way this is achieved, is by using Data Object Identifiers (URIs / IRIs that are generated by the Linked Data deployment platform) in the strict sense i.e. Data Identity (URI) is separated from Data Address (URL). Thus, you can reference a Person Data Object (aka an instance of a Person Class) in your HREF and the HTTP Server returns a Description of the Data Object via a Document (again, an information resource). A document containing the Description of a Data Object typically contains HREFs to other Data Objects that expose the Attributes and Relationships of the initial Person Data Object, and it this collection of Data Objects that is technically called a "Graph" -- which is what RDF models.

What I describe above is basic stuff for anyone that's familiar with Object Database or Distributed Objects technology and concepts.

URI and URL confusion

The Linked Document Web is a collection of physical resources that traverse the Web Information Bus in palatable format i.e documents. Thus, Document Object Identity and Document Object Data Address can be the same thing i.e. a URL can serve as the ID/URI of a Document Data Object.

The Linked Data Web on the other hand, is a Distributed Object Database, and each Data Object must be uniquely defined, otherwise we introduce ambiguity that ultimately taints the Database itself (making incomprehensible to reasoning challenged machines). Thus we must have unique Object IDs (URIs / IRIs) for People, Places, Events, and other things that aren't Documents. Once we follow the time tested rules of Identity, People can then be associated with the things they create (blog posts, web pages, bookmarks, wikiwords etc). RDF is about expressing these graph model relationships while RDF serialization formats enables the information resources to transport these data object link ladden information resources to requesting User Agents.

Put in more succinct terms, all documents on the Web are compound documents in reality (e.g. mast contain a least an image these days). The Linked Data Web is about a Web where Data Object IDs (URIs) enable us to distill source data from the information contained in a compound document.

Examples:

1. <http://community.linkeddata.org/dataspace/person/kidehen2#this> - the ID (URI minted from URL via addition of #this) of a Data Object of Type Person that Identifies me. The Person definition I use comes from the FOAF vocabulary/schema/ontology/data dictionary
2. <http://community.linkeddata.org/dataspace/person/kidehen2> - the URI (also a URL) of a FOAF file that contains a description of the Data Object ID: <http://community.linkeddata.org/dataspace/person/kidehen2#this> (me)
3. As an information resource <http://community.linkeddata.org/dataspace/person/kidehen2> can be dispatched from an HTTP server to a User Agent in (X)HTML, RDF/XML, N3/Turtle representations via HTTP Content Negotiation (note: Look at the "Linked Data" tab to see one example of what Data Links facilitate re. Data Discovery and Exploration)
4. If I choose an Object ID of <http://community.linkeddata.org/dataspace/person/kidehen2/this> instead of <http://community.linkeddata.org/dataspace/person/kidehen2#this> then the HTTP Server should not return an information resource (i.e provide 200 OK response) when a User Agent requests a resource via HTTP using the URI: <http://community.linkeddata.org/dataspace/person/kidehen2/this>, because a Data Object ID (URI) and the Data Object Address (URL) cannot be the same when my Data Object isn't of Type Document; the sever has to use response code 303 to redirect the user agent to the URL of an information resource that matches the Content-type designated in the HTTP Request or determine representation based on it's own quality of service rules for the information resource associated with the Object ID (URI).

The degree of unobtrusiveness of new technology, concepts, or new applications of existing technology, is what ultimately determines eventual uptake and meme virulence (network effects). For a while, the Semantic Web meme was mired in confusion and general misunderstanding due to a shortage of practical use case scenario demos.

The emergence of the SPARQL Query Language has provided critical infrastructure for a number of products, projects, and demos, that now make the utility of the Semantic Web vision mush clearly via the simplicity of Linked Data, as exemplified by the following:

1. Linking Open Data Community - collection of People and Linked Data Spaces (across a variety of domains)
2. DBpedia - Ground zero for experiencing and comprehending Linked Data
3. OpenLink Data Spaces - a simple solution for creating Linked Data Web presence via from existing Web Data Sources (Blogs, Wikis, Shared Bookmarks, Tag Spaces, Web Sites, Social Networking Services, Web Services, Discussion Forums etc..)
4. OpenLink Virtuoso - a Universal Server for generating, managing, and deploying RDF Linked Data from SQL, XML, Web Services based data sources

Related

Ryan Tomayko's post titled: So, What Does "HREF" Stand For, Anyway

Elias Torre's post titled: The Web FTW

Cool URIs for the Semantic Web.

Daniel Lewis also penned an interesting post in response to Ian's, that actually triggered this post.

I think definition time has long expired re. the Web's many interaction dimensions, evolutionary stages, and versions.

On my watch it's simply demo / dog-food time. Or as Dan Brickley states: Just Show It.

Below, I've created a tabulated view of the various lanes on the Web's Information Super Highway. Of course, this is a Linked Data demo should you be interested in the universe of data exposed via the links embedded in this post :-)

Now I can simply state the following using Linked Data (hyperdata) links:

1. Universal Data Access Drivers Suite (UDA) for ODBC, JDBC, ADO.NET, OLE-DB, and XMLA
2. OpenLink Data Spaces
3. Virtuoso

We no longer have to explain (repeatedly) why our drivers exist in Express, Lite, and Multi-Tier Edition formats, or why you ultimately need Multi-Tier Drivers over Single Tier Drivers (Express or Lite Editions) since you ultimately heed high-performance, data encryption, and policy based security across each of the data access driver formats.

OpenLink Data Spaces (ODS) now officially supports:

• Attention Profiling Markup Language (APML).
• Meaning of a Tag (MOAT) in conjunction with Simple Knowledge Organisation System (SKOS) and Social-Semantic Cloud of Tags (SCOT).
• OAuth - an Open Authentication Protocol

Which means that OpenLink Data Spaces support all of the main standards being discussed in the DataPortability Interest Group!

APML Example:

All users of ODS automatically get a dynamically created APML file, for example: APML profile for Kingsley Idehen

The URI for an APML profile is: http://myopenlink.net/dataspace/<ods-username>/apml.xml

Meaning of a Tag Example:

All users of ODS automatically have tag cloud information embedded inside their SIOC file, for example: SIOC for Kingsley Idehen on the Myopenlink.net installation of ODS.

But even better, MOAT has been implemented in the ODS Tagging System. This has been demonstrated in a recent test blog post by my colleague Mitko Iliev, the blog post comes up on the tag search: http://myopenlink.net/dataspace/imitko/weblog/Mitko%27s%20Weblog/tag/paris

Which can be put through the OpenLink Data Browser:

• OpenLink Data Browser with Mitko Iliev’s Paris Blog Tag

OAuth Example:

OAuth Tokens and Secrets can be created for any ODS application. To do this:

1. you can log in to MyOpenlink.net beta service, the Live Demo ODS installation, an EC2 instance, or your local installation
2. then go to ‘Settings’
3. and then you will see ‘OAuth Keys’
4. you will then be able to choose the applications that you have instantiated and generate the token and secret for that app.

Related Document (Human) Links

• OpenLink Data Spaces Official Page
• OpenLink Software Page
• OpenLink Data Spaces Wikipedia Page
• Attention Profiling Markup Language Project Website
• Meaning of a Tag Project Website
• Simple Knowledge Organisation Systems Project Website
• Social-Semantic Cloud of Tags Project Website
• OAuth Protocol Website
• DataPortability.org Website
• Semantically Interlinked Online Communities Project Website
  

Remember (as per my most recent post about ODS), ODS is about unobtrusive fusion of Web 1.0, 2.0, and 3.0+ usage and interaction patterns. Thanks to a lot of recent standardization in the Semantic Web realm (e.g SPARQL), we are now employ the MOAT, SKOS, and SCOT ontologies as vehicles for Structured Tagging.

Structured Tagging?

This is how we take a key Web 2.0 feature (think 2D in a sense), bend it over, to create a Linked Data Web (Web 3.0) experience unobtrusively (see earlier posts re. Dimensions of Web). Thus, nobody has to change how they tag or where they tag, just expose ODS to the URLs of your Web 2.0 tagged content and it will produce URIs (Structured Data Object Identifiers) and a lnked data graph for your Tags Data Space (nee. Tag Cloud). ODS will construct a graph which exposes tag subject association, tag concept alignment / intended meaning, and tag frequencies, that ultimately deliver "relative disambiguation" of intended Tag Meaning (i.e. you can easily discern the taggers meaning via the Tags actual Data Space which is associated with the tagger). In a nutshell, the dynamics of relevance matching, ranking, and the like, change immensely without futile timeless debates about matters such as:

What's the Linked Data value proposition?

What's the Linked Data business model?

XML vs RDF

XQuery vs SPARQL

What's the Semantic Web Killer application?

We can just get on with demonstrating Linked Data value using what exists on the Web today. This is the approach we are deliberately taking with ODS.

Related Items

Stefano Mazzocch's response to Clay Shirky's 2005 talk titled: Ontology is Overrated: Links, Tags and Post-hoc Metadata

Tom Gruber's post titled: Ontology of Folksonomy: A Mash-up of Apples and Oranges

Tip: This post is best viewed via an RDF aware User Agent (e.g. a Browser or Data Viewer). I say this because the permalink of this post is a URI in a Linked Data Space (My Blog) comprised of more data than meets the eye (i.e. what you see when you read this post via a Document Web Browser) :-)

Web Server:

- HTTP 1.1 compliant Transparent content-negotiation in URL-rewrite rules for Linked Data Deployment.

RDF Data Management:

- New providers for the Jena, Sesame and Redland frameworks

- support for SPARQL INSERT and UPDATE via HTTP POST

- New SPARQL-BI extenstions that make Business Intelligence feasible via SPARQL

- new "rdf_sink" folder for handling HTTP PUTs into WebDAV that automatically sync with Quad Store.

- There are new Sponger (RDFizer) cartridges that map Amazon book-search results to the Biliographic Ontology, supports production of Linked Data from OAI, XBRL, and Yahoo finance data sources.

- HTTPS protocol support added to Sponger

- performance optimizations for SPARQL `DESCRIBE' and `CONSTRUCT', alongside general performance enhancements for RDF data set loading.

Core DBMS Engine:

- PHP hosting a module re-implemented as a Virtuoso plugin inline with otherlanguage hosting modules

- improved deadlock condtion management

- enhanced POP and FTP server side protocol implementations that allow larger data transfers.

Additional Information

- DBpedia URI

- Product Home Page

- Wikipedia Page

- Virtuoso 5.0.4 Press Release

What is SPARQL?

A query language for the burgeoning Structured & Linked Data Web (aka Semantic Web / Giant Global Graph). Like SQL, for the Relational Data Model, it provides a query language for the Graph based RDF Data Model.

It's also a REST or SOAP based Web Service that exposes SPARQL access to RDF Data via an endpoint.

In addition, it's also a Query Results Serialization format that includes XML and JSON support.

Why is it Important?

It brings important clarity to the notion of the "Web as a Database" by transforming existing Web Sites, Portals, and Web Services into bona fide corpus of Mesh-able (rather than Mash-able) Data Sources. For instance, you can perform queries that join one or more of the aforementioned data sources in exactly the same manner (albeit different syntax) as you would one or more SQL Tables.

Example:

-- SPARQL equivalent of SQL SELECT * against my personal data space hosted FOAF file

SELECT DISTINCT ?s ?p ?o
FROM <http://myopenlink.net/dataspace/person/kidehen> 
WHERE {?s ?p ?o}

-- SPARQL against my social network -- Note: My SPARQL will be beamed across all of contacts in the social networks of my contacts as long as they are all HTTP URI based within each data space

PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT DISTINCT ?Person
FROM <http://myopenlink.net/dataspace/person/kidehen>
WHERE {?s a foaf:Person; foaf:knows ?Person}

Note: you can use the basic SPARQL Endpoint, SPARQL Query By Example, or SPARQL Query Builder Demo tool to experiment with the demonstration queries above.

How Do I use It?

SPARQL is implemented by RDF Data Management Systems (Triple or Quad Stores) just as SQL is implemented by Relational Database Management Systems. The aforementioned data management systems will typically expose SPARQL access via a SPARQL endpoint.

Where are it's implementations?

A SPARQL implementors Testimonial page accompanies the SPARQL press release. In addition the is a growing collection of implementations on the ESW Wiki Page for SPARQL compliant RDF Triple & Quad Stores.

Is this really a big deal?

Yes! SPARQL facilitates an unobtrusive manifestation of a Linked Data Web by way of natural extension of the existing Document Web i.e these Web enclaves co-exist in symbiotic fashion.

As DBpedia very clearly demonstrates, Linked Data makes the Semantic Web demonstrable and much easier to comprehend. Without SPARQL there would be no mechanism for Linked Data deployment, and without Linked Data there is no mechanism for Beaming Queries (directly or indirectly) across the Giant Global Graph of data hosted by Social Networks, Shard Bookmarks Services, Weblogs, Wikis, RSS/Atom/OPML feeds, Photo Galleries and other Web accessible Data Sources (Data Spaces).

Related items

Cool URIs

Publishing Linked Data Tutorial

Detailed SPARQL Query Examples using SIOC Data Spaces

Detailed SPARQL Query Examples using FOAF Data Spaces

Information overload and Data Portability are two of the most pressing and imminent challenges affecting every individual connected to the global village exposed by the Internet and World Wide Web. I wrote an earlier post titled: Why We Need Linked Data that shed light on frequently overlooked realities about the Document Web.

The real Killer application of the Semantic Web (imho) is Linked Data (or Hyperdata), just as the killer application of the Document Web was Linked Documents (Hyperlinks). Linked Data enables human users (indirectly) and software agents (directly in response to human instruction) to traverse Web Data Spaces (Linked Data enclaves within the Giant Global Graph).

Semantic Web applications (conduits between humans and agents) that take advantage of Linked Data include:

DBpedia - General Knowledge sourced from Wikipedia and a host of other Linked Data Spaces.

Various Linked Data Browsers: Zitgist Data Viewer, OpenLink RDF Browser, DISCO Browser, and TimBL's Tabulator.

zLknks - Linked Data Lookup technology for Web Content Publishing systems (note: more to come on this in a future post).

OpenLink Data Spaces - a solution for Data Portability via a Linked Data Junction Box for Web 1.0 ((X)HTML Document Webs), 2.0 (XML Web Services based Content Publishing, Content Syndication, and Aggregation), and 3.0 (Linked Data) Data Spaces. Thus, via my URI (when viewed through a Linked Data Browser/Viewer) you can traverse my Data Space (i.e my Linked Data Graph) generated by the following activities:

Blog Posts publishing

My RSS & Atom Content Subscriptions (what used to be called a "Blogroll")

My Bookmarks (from my Desktop and Del.icio.us)

and other things I choose to share with the public via the Web

Virtuoso - a Universal Server Platform that includes RDF Data Management, RDFization Middleware, SQL-RDF Mapping, RDF Linked Data Deployment, alongside a hybrid/multi-model, virtual/federated data service in a single product offering.

1. Web 2.0 is fundamentally about Web Services as the prime vehicle for interactions across "points of Web presence"
2. Facebook is a Web 2.0 hosted service for social networking that provides Web Services APIs for accessing data in the Facebook data space. You have to do so "on the fly" within clearly defined constraints i.e you can interact with data across your social network via Facebook APIs, but you cannot cache the data (perform an export style dump of the data)
3. Facebook is a main driver of the term: "social graph", but their underlying data model is relational and the Web Services response (data you get back) doesn't return a data graph, instead it returns an tree (i.e XML)
4. Scoble's had a number of close encounters with Linked Data Web | Semantic Data Web | Web 3.0 aficionados in various forms throughout 2007, but still doesn't quite make the connection between Web Services APIs as part of a processing pipeline that includes structured data extraction from XML data en route to producing Data Graphs comprised of Data Objects (Entities) endowed with: Unique Identifiers, Classification or Categorization schemes, Attributes, and Relationships prescribed by one or more shared Data Dictionaries/Schemas/Ontologies
5. A global information bus that exposes a Linked Data mesh comprised of Data Objects, Object Attributes, and Object Relationships across "points of Web presence" is what TimBL described in 1998 (Semantic Web Roadmap) and more recently in 2007 (Giant Global Graph)
6. The Linked Data mesh (i.e Linked Data Web or GGG) is anchored by the use of HTTP to mint Location, Structure, and Value independent Object Identifiers called URIs or IRIs. In addition, the Linked Data Web is also equipped with a query language, protocol, and results serialization format for XML and JSON called: SPARQL.

So, unlike Scoble, I am able to make my Facebook Data portable without violating Facebook rules (no data caching outside Facebook realm) by doing the following:

1. Use an RDFizer for Facebook to convert XML response data from Facebook Web Services into RDF "on the fly" Ensure that my RDF is comprised of Object Identifiers that are HTTP based and thereby dereferencable (i.e. I can use SPARQL to unravel the Linked Data Graph in my Facebook data space)
2. The act of data dereferencing enables me to expose my Facebook Data as Linked Data associated with my Personal URI
3. This interaction only occurs via my data space and in all cases the interactions with data work via my RDFizer middleware (e.g the Virtuoso Sponger) that talks directly to Facebook Web Services.

In a nutshell, my Linked Data Space enables you to reference data in my data space via Object Identifiers (URIs), and some cases the Object IDs and Graphs are constructed on the fly via RDFization middleware.

Here are my URIs that provide different paths to my Facebook Data Space:

Personal URI

My Facebook Data Space (best viewed via a Linked Data Browser/Viewer session)

My Facebook Photo Gallery -- WWW2007 Photo Collection (also best viewed via a Linked Data Browser/Viewer session)

To conclude, 2008 is clearly the inflection year during which we will final unshackle Data and Identity from the confines of "Web Data Silos" by leveraging the HTTP, SPARQL, and RDF induced virtues of Linked Data.

Related Posts:

1. 2008 and the Rise of Linked Data
2. Scoble Right, Wrong, and Beyond
3. Scoble interviewing TimBL (note to Scoble: re-watch your interview since he made some specific points about Linked Data and URIs that you need to grasp)
4. Prior Blog posts my this Blog Data Space that include the literal patterns: Scoble Semantic Web

Writing a JDBC Driver for SPARQL is a little overkill. OpenOffice.org simply needs to make XML or Web Data (HTML, XHTML, and XML) bonafide data sources within its "Pivot Table" functionality realm. Then all that would then be required is a SPARQL SELECT Query transported via the SPARQL Protocol with results sent back using the SPARQL XML results serialization format (all part of a single SPARQL Protocol URL).

Excel successfully consumes the following information resource URI: http://tinyurl.com/yvoccj (a tiny url for a SPARQL SELECT against my FOAF file).

Alternatively, and currently achievable, you could simply use SPASQL (SPARQL within SQL) using a DBMS engine that supports SQL, SPARQL, and SPARQL e.g. Virtuoso.

Virtuoso SPASQL support is exposed via it's ODBC and/or JDBC Drivers. Thus you can do things such as:

1. Use a SPARQL Query in the FROM CLAUSE of a SQL statement
2. Execute SPARQL via SQL processor by prepending SPARQL query text with the literals "sparql"

BTW - My News Years Resolution: get my act together and shrink the ever increasing list of "simple & practical Virtuoso use case demos" on my todo which now spans all the way back to 2006 :-(

First off, I am going to focus on the Semantic Data Web aspect of the overall Semantic Web vision (a continuum) as this is what we have now. I am also writing this post as a deliberate contribution to the discourse swirling around the real topic: Semantic Web Value Proposition.

Situation Analysis

We are in the early stages of the long anticipated Knowledge Economy. That being the case, it would be safe to assume that information access, processing, and dissemination are of utmost importance to individuals and organizations alike. You don't produce knowledge in a vacum! Likewise, you can produce Information in a vacum, you need Data.

The Semantic Data Web's value to Individuals

Increasingly, Blogs, Wikis, Shared Bookmarks, Photo Galleries, Discussion Forums, Shared Calendars and the like, have become invaluable tools for individual and organizational participation in Web enabled global discourse (where a lot of knowledge is discovered). These tools, are typically associated with Web 2.0, implying Read-Write access via Web Services, centralized application hosting, and data lock-in (silos).

The reality expressed above is a recipe for "Information Overload" and complete annihilation of ones effective pursuit and exploitation of knowledge due "Time Scarcity" (note: disconnecting is not an option). Information abundance is inversely related to available processing time (for humans in particular). In my case for instance, I was actively subscribed to over 500+ RSS feeds in 2003. As of today, I've simply stopped counting, and that's just my Weblog Data Space. Then add to that, all of the Discussions I track across Blogs, wikis, message boards, mailing lists, traditional usnet discussion forumns, and the like, and I think you get the picture.

Beyond information overload, Web 2.0 data is "Semi-Structured" by way of it's dominant data containers ((X)HTML, RSS, Atom documents and data streams etc.) lacking semantics that formally expose individual data items as distinct entities, endowed with unambiguous naming / identification, descriptive attributes (a type of property/predicate), and relationships (a type of property/predicate).

Devise a standard for Structured Data Semantics that is compatible with the Web Information BUS.

Produce structured data (entities, entity types, entity relationships) from Web 1.0 and Web 2.0 resources that already exists on the Web such that individual entities, their attributes, and relationships are accessible and discernible to software agents (machines).

Once the entities are individually exposed, the next requirement is a mechanism for selective access to these entities i.e. a query language.

Semantic Data Web Technologies that facilitate the solution described above include:

RDF - Data Model for structured data

RDF/XML - A serialization format for RDF based structured data

N3 / Turtle - more human friendly serialization formats for RDF based structured data

GRDDL - enables association between XHTML pages and XSLT stylesheets that facilitates loosely coupled "on the fly" extraction of RDF from non RDF documents

RDFa - enables document publishers or viewers (i.e those repurposing or annotating) to embed structured data into existing XHTML documents

eRDF - another option for embedding structured RDF data within (X)HTML documents

RDF Middleware - typically incorporating GRDDL, RDFa, eRDF, and custom extraction and mapping as part of a structured data production pipeline

Use of URIs or IRIs for uniquely identifying physical (HTML Documents, Image Files, Multimedia Files etc..) and abstract (People, Places, Music, and other abstract things).

SPARQL Query Language - the SQL analog of the Semantic Data Web that enables query constructs that target named entities, entity attributes, and entity relationships

SPARQL Protocol - a REST or SOAP style Web Service for transporting SPARQL Queries to Structured Data Sources.

SPARQL Results Serialization Formats - query results serialization formats that includes XML(sparql+xml) and JSON.

The Semantic Data Web's value to Organizations

Organizations are rife with a plethora of business systems that are built atop a myriad of database engines, sourced from a variety of DBMS vendors. A typical organization would have a different database engine, from a specific DBMS vendor, underlying critical business applications such as: Human Resource Management (HR), Customer Relationship Management (CRM), Accounting, Supply Chain Management etc. In a nutshell, you have DBMS Engines, and DBMS Schema heterogeneity permeating the IT infrastructure of organizations on a global scale, making Data & Information Integration the biggest headache across all IT driven organizations.

Alleviation of the pain (costs) associated with Data & Information Integration.

A dexterous data model (RDF) that enables the construction of conceptual views of disparate data sources across an organization based on existing web architecture components such as HTTP and URIs.

Existing middleware solutions that facilitate the exposure of SQL DBMS data as RDF based Structured Data include:

Virtuoso's Meta Schema Language for RDF Views of SQL Data (also see the Virtuoso SQL-RDF Technical White Paper)

D2RQ

DataGrid

Others

BTW - There is an upcoming W3C Workshop covering the integration of SQL and RDF data.

Conclusion

The Semantic Data Web is here, it's value delivery vehicle is the URI. The URI is a conduit to Interlinked Structured Data (RDF based Linked Data) derived from existing data sources on the World Wide Web alongside data continuously injected into the Web by organizations world wide. Ironically, the Semantic Data Web only platform that crystallizes the: Information at Your Fingertips vision, without development environment, operating system, application, or database lock-in. You simply click on a Linked Data URI and the serendipitous exploration and discovery of data commences.

The unobtrusive emergence of the Semantic Data Web is a reflection of the soundness of the underlying Semantic Web vision.

If you are excited about Mash-ups then your are a Semantic Web enthusiast and benefactor in the making, because you only "Mash" (brute force data extraction and interlinking) because you can't "Mesh" (natural data extraction and interlinking). Likewise, if you are a social-networking, open social-graph, or portable social-network enthusiast, then you are also a Semantic Data Web benefactor and enthusiasts, because your "values" (yes, the values associated with the properties that define you e.g your interests etc) are the fundamental basis for portable, open, social-networking, which is what the Semantic Data Web hands to you on a platter without compromise (i.e. data lock-in or loss of data ownership).

Read/WriteWeb via the OpenLink Data Web Browser (click on the different viewing tabs to see what structured data exploitation in action)

Read/WriteWeb via the Zitgist Data Web Browser

DBpedia (*note: I deliberately use DBpedia URIs in my posts where I would otherwise have used a Wikipedia article URI*)

Zitgist zLinks - Mike Bergman's Blog Post also demonstrating zLinks

My Comments:

Hyperdata is short for HyperLinked Data :-) The same applies to Linked Data. Thus, we have two literal labels for the same core Concept. HTTP is the enabling protocol for "Hyper-linking" Documents and associated Structured Data via the World Wide Web (Web for short). Data Links associated with Structured Data contained in, or hosted by, Documents on the Web.

RDFa, eRDF, GRDDL, SPARQL Query Language, SPARQL Protocol (SOAP or REST service), SPARQL Results Serializations (XML or JSON) collectively provide a myriad of unobtrusive routes to structured data embedded within, or associated with, existing Web Documents.

As Danny already states, ontologies are not prerequisites for producing structured data using the RDF Data Model. They simply aid the ability to express one's self clearly (i.e. no repetition or ambiguity) across a broad audience of machines (directly) and their human masters (indirectly).

Using the crux of this post as the anecdote: The Semantic Data Web would simplify the process of claiming and/or proving that Linked Data and Hyperdata describe the same concept. It achieves this by using Triples (Subject, Predicate, Object) expressed in various forms (N3, Turtle, RDF/XML etc.) to formalize claims in a form palatable to electronic agents (machines) operating on behalf of Humans. In a nutshell, this increases human productive by completely obliterates the erstwhile exponential costs of discovering data, information, and knowledge.

BTW - for full effect, view this post (i.e. cut and paste the Permalink URI of this post, below) into an RDF Browser such as:

- OpenLink RDF Browser

- Zitgist Browser

- DISCO

- Tabulator

New Semantic Data Web related features and enhancements include:

* A Javascript-based Fresnel processor enabling declarative RDF-based display templates for RDF Data Sources

* An XSLT template for generating HTML pages from the Fresnel processor's XML output

* Interactive Query Builder for SPARQL (iSPARQL). This version of the iSPARQL application includes support for INSERTs and DELETEs

* Enhanced Javascript-based N3/Turtle parser

* New Navigator viewer panel for RDF Browser.

*Project Home Page

*Source Code

*Live Features Demonstrations.

On different, but related, thread, Mike Bergman recently penned a post titled: What is the Structured Web?. Both of these public contributions shed light on the "Information BUS" essence of the World Wide Web by describing the evolving nature of the payload shuttled by the BUS.

What is an Information BUS?

Middleware infrastructure for shuttling "Information" between endpoints using a messaging protocol.

The Web is the dominant Information BUS within the Network Computer we know as the "Internet". It uses HTTP to shuttle information payloads between "Data Sources" and "Information Consumers" - what happens when we interact with Web via User Agents / Clients (e.g Browsers).

What are Web Information Payloads?

HTTP transported streams of contextualized data. Hence the terms: "Information Resource" and "Non Information" when reading material related to http-range-14 and Web Architecture. For example, an (X)HTML document is a specific data context (representation) that enables us to perceive, or comprehend, a data stream originating from a Web Server as a Web Page. On the other hand, if the payload lacks contextualized data, a fundamental Web requirement, then the resource is referred to as a "Non Information" resource. Of course, there is really no such thing as a "Non Information" resource, but with regards to Web Architecture, it's the short way of saying: "the Web Transmits Information only". That said, I prefer to refer to these "Non Information" resources as "Data Sources", are term well understood in the world of Data Access Middleware (ODBC, JDBC, OLEDB, ADO.NET etc.) and Database Management Systems (Relational, Objec-Relational, Object etc).

Examples of Information Resource and Data Source URIs:

http://demo.openlinksw.com/Northwind/Customer/ALFKI (Information Resource)

http://demo.openlinksw.com/Northwind/Customer/ALFKI#this (Data Source)

Explanation: The Information Resource is a conduit to the Entity identified by Data Source (an entity in my RDF Data Space that is the Subject or Object of one of more Triple based Statements. The triples in question can that can be represented as an RDF resource when transmitted over the Web via an Information Resource that takes the form of a SPARQL REST Service URL or a Physical RDF based Information Resource URL).

What about Structured Data?

Prior to the emergence of the Semantic Data Web, the payloads shuttled across the Web Information BUS comprised primarily of the following:

1. HTML - Web Resource with presentation focused structure (Web 1.0 dominant payload form)
2. XML - Web Resource with structure that separates presentation and data (Web 2.0's dominant payload form).

The Semantic Data Web simply adds RDF to the payload formats that shuttle the Web Information BUS. RDF addresses formal data structure which XML doesn't cover since it is semi-structured (distinct data entities aren't formally discernible). In a nutshell, an RDF payload is basically a conceptual model database packaged as an Information Resource. It's comprised of granular data items called "Entities", that expose fine grained properties values, individual and/or group characteristics (attributes), and relationships (associations) with other Entities.

Where is this all headed?

The Web is in the final stages of the 3rd phase of it's evolution. A phase characterized by the shuttling of structured data payloads (RDF) alongside less data oriented payloads (HTML, XHTML, XML etc.). As you can see, Linked Data and Structured Data are both terms used to describe the addition of more data centric payloads to the Web. Thus, you could view the process of creating a Structured Web of Linked Data as follows:

1. Identify or Create Structured Data Sources
2. Name these Data Sources using Data Source URIs
3. Expose Structured Data Sources to the Web as Linked Data using Information Resource (conduit) URIs

Conclusions

The Semantic Data Web is an evolution of the current Web (an Information Space) that adds structured data payloads (RDF) to current, less data oriented, structured payloads (HTML, XHTML, XML, and others).

The Semantic Data Web is increasingly seen as an inevitability because it's rapidly reaching the point of critical mass (i.e. network effect kick-in). As a result, Data Web emphasis is moving away from: "What is the Semantic Data Web?" To: "How will Semantic Data Web make our globally interconnected village an even better place?", relative to the contributions accrued from the Web thus far. Remember, the initial "Document Web" (Web 1.0) bootstrapped because of the benefits it delivered to blurb-style content publishing (remember the term electronic brochure-ware?). Likewise, in the case of the "Services Web" (Web 2.0), the bootstrap occurred because it delivered platform independence to Web Application Developers - enabling them to expose application logic behind Web Services. It is my expectation that the Data Integration prowess of the Data Web will create a value exchange realm for data architects and other practitioners from the database and data access realms.

Related Items

1. Mike Bergman's post about Semi-Structured Data
2. My Posts covering Structured and Un-Structured Containers

A vital component of the new Virtuoso release is the finalization of our SQL to RDF mapping functionality -- enabling the declarative mapping of SQL Data to RDF. Additional technical insight covering other new features (delivered and pending) is provided by Orri Erling, as part of a series of post-Banff posts.

Why is SQL to RDF Mapping a Big Deal?

A majority of the world's data (especially in the enterprise realm) resides in SQL Databases. In addition, Open Access to the data residing in said databases remains the biggest challenge to enterprises for the following reasons:

1. SQL Data Sources are inherently heterogeneous because they are acquired with business applications that are in many cases inextricably bound to a particular DBMS engine
2. Data is predictably dirty
3. DBMS vendors ultimately hold the data captive and have traditionally resisted data access standards such as ODBC (*trust me they have, just look at the unprecedented bad press associated with ODBC the only truly platform independent data access API. Then look at how this bad press arose..*)

Enterprises have known from the beginning of modern corporate times that data access, discovery, and manipulation capabilities are inextricably linked to the "Real-time Enterprise" nirvana (hence my use of 0.0 before this becomes 3.0).

In my experience, as someone whose operated in the data access and data integration realms since the late '80s, I've painfully observed enterprises pursue, but unsuccessfully attain, full control over enterprise data (the prized asset of any organization) such that data-, information-, knowledge-workers are just a click away from commencing coherent platform and database independent data drill-downs and/or discovery that transcend intranet, internet, and extranet boundaries -- serendipitous interaction with relevant data, without compromise!

Okay, situation analysis done, we move on..

At our most recent (12th June) monthly Semantic Web Gathering, I unveiled to TimBL and a host of other attendees a simple, but powerful, demonstration of how Linked Data, as an aspect of the Semantic Data Web, can be applied to enterprise data integration challenges.

Actual SQL to RDF Mapping Demo / Experiment

Hypothesis

Apparatus

Experiment / Demo

1. Declaratively map the Northwind SQL Schema to RDF using the Virtuoso Meta Schema Language (see: Virtuoso PL based Northwind_SQL_RDF script)
2. Start browsing the data by clicking on the URIs that represent the RDF Data Model Entities resulting from the SQL to RDF Mapping

Observations

1. Via a single Data Link click I was able to obtain specific information about the Customer represented by the URI "ALFKI" (act of URI Dereferencing as you would an Object ID in an Object or Object-Relational Database)
2. Via a Dynamic Data Page I was able to explore all the entity relationships or specific entity data (i.e Exploratory or Entity specific dereferencing) in the Northwind Data Space
3. I was able to perform similar exploration (as per item 2) using our OpenLink Browser.

Conclusions

The vision of data, information, or knowledge at your fingertips is nigh! Thanks to the infrastructure provided by the Semantic Data Web (URIs, RDF Data Model, variety of RDF Serialization Formats[1][2][3], and Shared Data Dictionaries / Schemas / Ontologies [1][2][3][4][5]) it's now possible to Virtualize enterprise data from the Physical Storage Level, through the Logical Data Management Levels (Relational), up to a Concrete Conceptual Model (Graph) without operating system, development environment or framework, or database engine lock-in.

Next Steps

We produce a shared ontology for the CRM and Business Reporting Domains. I hope this experiment clarifies how this is quite achievable by converting XML Schemas to RDF Data Dictionaries (RDF Schemas or Ontologies). Stay tuned :-)

Also watch TimBL amplify and articulate Linked Data value in a recent interview.

Other Related Matters

To deliver a mechanism that facilitates the crystallization of this reality is a contribution of boundless magnitude (as we shall all see in due course). Thus, it is easy to understand why even "her majesty", the queen of England, simply had to get in on the act and appoint TimBL to the "British Order of Merit" :-)

Note: All of the demos above now work with IE & Safari (a "remember what Virtuoso is epiphany") by simply putting Virtuoso's DBMS hosted XSLT engine to use :-) This also applies to my earlier collection of demos from the Hello Data Web and other Data Web & Linked Data related demo style posts.

Note: the enhanced hyperlink (typed data link) lookup presents options to perform an Explore (all data about subject across Domains in the data space i.e. data links to and from Subject), Dereference (specific data in the Subject's Domain i.e. data links originating from subject).

1. Diana Ross
2. Paul McCartney
3. The Beatles
4. Madonna

I built these Linked Data Pages by simply doing the following:

1. Open up our OAT based iSPARQL (Interactive SPARQL Query By Example) Tool
2. Paste a URI of Interest into the Data Source URI input field
3. Execute the Query (hitting the ">" button)
4. Saving the Query to WebDAV as a Linked Data Page (or what I initial called Dynamic Data Web pages in my Hello Data Web series of posts).
5. Share your Data, Information, Knowledge with others via URIs (as shown in the section above).

Now that broader understanding of the Semantic Data Web is emerging, I would like to revisit the issue of "Data Spaces".

A Data Space is a place where Data Resides. It isn't inherently bound to a specific Data Model (Concept Oriented, Relational, Hierarchical etc..). Neither is it implicitly an access point to Data, Information, or Knowledge (the perception is purely determined through the experiences of the user agents interacting with the Data Space.

A Web Data Space is a Web accessible Data Space.

Real world example:

Today we increasing perform one of more of the following tasks as part of our professional and personal interactions on the Web:

1. Blog via many service providers or personally managed weblog platforms
2. Create Event Calendars via Upcoming.com and Eventful
3. Maintain and participate in Social Networks (e.g. Facebook, Orkut, MySpace)
4. Create and Participate in Discussions (note: when you comment on blogs or wikis for instance, you are participating in, or creating, a conversation)
5. Track news by subscribing to RSS 1.0, RSS 2.0, or Atom Feeds
6. Share Bookmarks & Tags via Del.icio.us and other Services
7. Share Photos via Flickr
8. Buy, Review, or Search for books via Amazon
9. Participates in auctions via eBay
10. Search for data via Google (of course!)

John Breslin has nice a animation depicting the creation of Web Data Spaces that drives home the point.

Unfortunately, what isn't as obvious to many netizens, is the fact that each of the activities above results in the creation of data that is put into some context by you the user. Even worse, you eventually realize that the service providers aren't particularly willing, or capable of, giving you unfettered access to your own data. Of course, this isn't always by design as the infrastructure behind the service can make this a nightmare from security and/or load balancing perspectives. Irrespective of cause, we end up creating our own "Data Spaces" all over the Web without a coherent mechanism for accessing and meshing these "Data Spaces".

Data Spaces on the Web that provide granular access to RDF Data.

Short History

In anticipation of this the "Web Data Silo" challenge (an issue that we tackled within internal enterprise networks for years) we commenced the development (circa. 2001) of a distributed collaborative application suite called OpenLink Data Spaces (ODS). The project was never released to the public since the problems associated with the deliberate or inadvertent creation of Web Data silos hadn't really materialized (silos only emerged in concreted form after the emergence of the Blogosphere and Web 2.0). In addition, there wasn't a clear standard Query Language for the RDF based Web Data Model (i.e. the SPARQL Query Language didn't exist).

Today, ODS is delivered as a packaged solution (in Open Source and Commercial flavors) that alleviates the pain associated with Data Space Silos that exist on the Web and/or behind corporate firewalls. In either scenario, ODS simply allows you to create Open and Secure Data Spaces (via it's suite of applications) that expose data via SQL, RDF, XML oriented data access and data management technologies. Of course it also enables you to integrates transparently with existing 3rd party data space generators (Blogs, Wikis, Shared Bookmrks, Discussion etc. services) by supporting industry standards that cover:

1. Content Publishing - Atom, Moveable Type, MetaWeblog, Blogger protocols
2. Content Syndication Formats - RSS 1.0, RSS 2.0, Atom, OPML etc.
3. Data Management - SQL, RDF, XML, Free Text
4. Data Access - SQL, SPARQL, GData, Web Services (SOAP or REST styles), WebDAV/HTTP
5. Semantic Data Web Middleware - GRDDL, XSLT, SPARQL, XPath/XQuery, HTTP (Content Negotiation) for producing RDF from non RDF Data ((X)HTML, Microformats, XML, Web Services Response Data etc).

Thus, by installing ODS on your Desktop, Workgroup, Enterprise, or public Web Server, you end up with a very powerful solution for creating Open Data access oriented presence on the "Semantic Data Web" without incurring any of the typically assumed "RDF Tax".

Naturally, ODS is built atop Virtuoso and of course it exploits Virtuoso's feature-set to the max. It's also beginning to exploit functionality offered by the OpenLink Ajax Toolkit (OAT).

Some Definitions (as per usual):

RDF Middleware (as defined in this context) is about producing RDF from non RDF Data Sources. This implies that you can use non RDF Data Sources (e.g. (X)HTML Web Pages, (X)HTML Web Pages hosting Microformats, and even Web Services such as those from Google, Del.icio.us, Flickr etc..) as Semantic Web Data Source URIs (pointers to RDF Data).

In this post I would like to provide a similar perspective on this ability to treat non RDF as RDF from RDF Browser perspective.

First off, what's an RDF Browser?

An RDF Browser is a piece of technology that enables you to Browse RDF Data Sources by way of Data Link Traversal. The key difference between this approach and traditional browsing is that Data Links are typed (they possess inherent meaning and context) whereas traditional links are untyped (although universally we have been trained to type them as links to Blurb in the form of (X)HTML pages or what is popularly called "Web Content".).

There are a number of RDF Browsers that I am aware off (note: pop me a message directly of by way of a comment to this post if you have a browser that I am unaware of), and they include (in order of creation and availability):

1. Tabulator
2. DISCO - Hyperdata Browser
3. OpenLink Ajax Toolkit's RDF Browser (a component of the OAT Javascript Toolkit)

Each of the browsers above can consume the services of Triplr or the Virtuoso Sponger en route to unveiling a RDF Data that is traversable via URI dereferencing (HTTP GETing the data exposed by the Data Pointer). Thus you can cut&paste the following into each of the aforementioned RDF Browsers:

1. Triplr's RDF Data (Triples) extractions from Dan Connolly's Home Page
2. The Virtuoso Sponger's RDF Data (Triples) extractions from Dan Connolly's Home Page

Since we are all time challenged (naturally!) you can also just click on these permalinks for the OAT RDF Browser demos:

1. Permalink for Triplr's RDF Data (Triples) extractions from Dan Connolly's Home Page
2. Permalink for the Virtuoso Sponger's RDF Data (Triples) extractions from Dan Connolly's Home Page

1. Messaging Issues - misconceptions and misrepresentations (e.g intermingling or RDF the Data Model and RDF/XML one of several serialization formats)
2. RDF Data Availability
3. Generating RDF from non RDF Data ("RDF Tax" eradication)
4. Querying RDF Data Sources

Quick Definitions:

Mashups - Brute force joining of disparate Web Data

Meshups - Natural joining of disparate Web Data

Reasons for the distinction:

Mashups are Data Model oblivious.

Meshups are Data Model driven.

Examples:

Mashups are based on RSS 2.0 most of the time (RSS 2.0 is at best a Tree Structure that contains untyped or meaning challenged links.

Meshups are RDF based and the data is self describing since the links are typed (posses inherent meaning thereby providing context).

So what? You may be thinking.

For starters, I can quite easily Mesh data from Googlebase (which emits RSS 2.0 or Atom) and other data sources with the Mapping Services from Yahoo!

I can achieve this in minutes without writing a single line of code. I can do it because of the Data Model prowess of RDF (self-describing instance-data), the data interchange and transformation power of XML and XSLT respectively, the inherent power of XML based Web Services (REST or SOAP), and of course, having a Hybrid Server product like Virtuoso at my disposal that delivers a cross platform solution for exploiting all of these standards coherently.

I can share the self-describing describing data source that serves my Meshup. Try reusing the data presented by a Mashup via the same URL that you used to locate Mashup to get my drift.

Demo Links:

1. Googlebase Query URL as an RDF Data Source
2. Perform a simple Data Mesh by adding (via link copy and paste) this Upcoming.org Query Services URL for Ajax Events to the RDF Browsers list of Data Sources (paste into the Data Source URI input field).

What does this all mean?

"Context" is the catalyst of the burgeoning Data Web (Semantic Web Layer - 1). It's the emerging appreciation of "Context" that is driving the growing desire to increment Web versions from 2.0 to 3.0. It also the the very same "Context" that has been a preoccupation of Semantic Web vision since its inception.

The journey towards a more Semantic Web is all inclusive (all "ANDs" and no "ORs" re. participation).

The Semantic Web is self-annotating. Web 2.0 has provided a huge contribution to the self annotation effort: on the Web we now have Data Spaces for Bookmarks (e.g del.icio.us), Image Galleries ( e.g Flickr), Discussion Forums (remember those comments associated with blog posts? ditto the pingbacks and trackbacks?), People Profiles (FOAF, XFN, del.icio.us, and those crumbling walled-gardens around many Social Networks), and more..

A Web without granular access to Data is simply not a Web worth having (think about the menace of click-fraud and spam).

PREFIX dbpedia: <http://dbpedia.org/>
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?name ?birth ?death
FROM <http://dbpedia.org>
WHERE {
   ?person dbpedia:birthplace <http://dbpedia.org/resource/Berlin> .
   ?person dbpedia:birth ?birth .
   ?person foaf:name ?name .
   ?person dbpedia:death ?death
   FILTER (?birth < "1900-01-01"^^xsd:date and bif:contains (?name,
'otto')) .
}
ORDER BY ?name

You can test further using our SPARQL Endpoint for DBpedia or via the DBPedia bound Interactive SPARQL Query Builder or just click *Here* for results courtesy of the SPARQL Protocol (REST based Web Service).

Note: This is in-built functionality as Virtuoso has possessed Full Text Indexing since 1998-99. This capability applies to physical and virtual graphs managed by Virtuoso.

A per usual, there is more to come as we now have a nice intersection point for SPARQL and XQuery/XPath since Triple Objects (the Literal variety) can take the form of XML Schema based Complex Types :-) A point I alluded too in my podcast interview with Jon Udell last year (*note: mechanical turk based transcript is bad*). The point I made went something like this: "...you use SPARQL to traverse the typed links and then use XPath/XQuery for further granular access to the data if well-formed..."

Anyway, the podcast interview lead to this InfoWorld article titled: Unified Data Theory.

Rich Internet Applications ultimately enable intelligent processing of self-describing databases originating from data servers as demonstrated by these examples:

1. My Dynamic Data Web Start Page
2. Chris Bizer Data Space
3. Our RDF Browser (just enter a Web URI e.g http://sites.wiwiss.fu-berlin.de/suhl/bizer/foaf.rdf or http://www.openlinksw.com and then drill down; not Grandma's unobtrusive Data Web Navigator, but headed in that direction..)

XMP and microformats revisited: "

OAT: OpenAjax Alliance Compliant Toolkit: "

Ondrej Zara and his team at Openlink Software have created a Openlink Software JS Toolkit, known as OAT. It is a full-blown JS framework, suitable for developing
rich applications with special focus to data access.

OAT works standalone, offers vast number of widgets and has some rarely seen features, such as on-demand library loading (which reduces the total amount of downloaded JS code).

OAT is one of the first JS toolkits which show full OpenAjax Alliance conformance: see the appropriate wiki page and conformance test page.

There is a lot to see with this toolkit:

You can see some of the widgets in a Kitchen sink application

Sample data access applications:

• SQL Query By Example
• Forms designer
• DB Designer

OAT is Open Source and GPL’ed over at sourceforge and the team has recently managed to incorporate our OAT data access layer as a
module to dojo datastore.

(Via Ajaxian Blog.)

This is a corrected version of the initial post. Unfortunately, the initial post was inadvertently littered with invalid links :-( Also, since the original post we have released OAT 1.2 that includes integration of our iSPARQL QBE into the OAT Form Designer application.

Re. Data Access, It is important to note that OAT's Ajax Database Connectivity layers supports data binding to the following data source types:

1. RDF - via SPARQL (Query Language, Protocol, and Resultset Serialization formats: RDF/XML, RDF/N3, RDF/Turtle, XML, and JSON)
2. SQL - via XMLA (somewhat forgotten SOAP protocol for SQL Data Access that can sit atop ODBC, ADO.NET, OLE-DB, and even JDBC)
3. XML - via SOAP or REST style Web Services

1. Tabular Grids
2. Pivot Tables
3. TimeLines
4. Extended Anchor Tags
5. Map Service Controls (Google, Yahoo!, OpenLayers, Microsoft Visual Earth)
6. SVG based RDF Graph Control (Opera 9.x provides best viewing experience at the current time)

OAT also includes a number of prototype applications that are completely developed using OAT Controls and Libraries:

1. Visual SPARQL Query Builder
2. Visual SQL Query Builder
3. Web Forms Designer (includes Drag-Drop usage of Data Aware Controls etc.)
4. Visual DB Designer

Note: Pick "Local DSN" from page initialization dialog's drop-down list control when prompted

The Semantic Web is about granular exposure of the underlying web-of-data that fuels the World Wide Web. It models "Web Data" using a Directed Graph Data Model (back-to-the-future: Network Model Database) called RDF.

In line with contemporary database technology thinking, the Semantic Web also seeks to expose Web Data to architects, developers, and users via a concrete Conceptual Layer that is defined using RDF Schema.

The abstract nature of Conceptual Models implies that actual instance data (Entities, Attributes, and Relationships/Associations) occurs by way of "Logical to Conceptual" schema mapping and data generation that can involve a myriad of logical data sources (SQL, XML, Object databases, traditional web content, RSS/Atom feeds etc.). Thus, by implication, it is safe assume that the Semantic Web's construction is basically a Data Integration and exposure effort. The point that Stefano alludes to in the blog post excerpts that follow:

The semantic web is really just data integration at a global scale. Some of this data might end up being consistent, detailed and small enough to perform symbolic reasoning on, but even if this is the case, that would be such a small, expensive and fragile island of knowledge that it would have the same impact on the world as calculus had on deciding to invade Iraq.

The biggest problem we face right now is a way to 'link' information that comes from different sources that can scale to hundreds of millions of statements (and hundreds of thousands of equivalences). Equivalences and subclasses are the only things that we have ever needed of OWL and RDFS, we want to 'connect' dots that otherwise would be unconnected. We want to suggest people to use whatever ontology pleases them and then think of just mapping it against existing ones later. This is easier to bootstrap than to force them to agree on a conceptualization before they even know how to start!

Additional insightful material from Stefano:

1. A No-Nonsense Guide to Semantic Web Specs for XML People [Part I]
2. A No-nonsense Guide to Semantic Web Specs for XML People [Part II]

Benjamin Nowack also chimes into this conversation via his simple guide to understanding Data, Information, and Knowledge in relation so the Semantic Web.

SPARQL (query language for the Semantic Web) basically enables me to query a collection of typed links (predicates/properties/attributes) in my Data Space (ODS based of course) without breaking my existing local bookmarks database or the one I maintain at del.icio.us.

I am also demonstrating how Web 2.0 concepts such as Tagging mesh nicely with the more formal concepts of Topics in the Semantic Web realm. The key to all of this is the ability to generate RDF Data Model Instance Data based on Shared Ontologies such as SIOC (from DERI's SIOC Project) and SKOS (again showing that Ontologies and Folksonomies are complimentary).

This demo also shows that Ajax also works well in the Semantic Web realm (or web dimension of interaction 3.0) especially when you have a toolkit with Data Aware controls (for SQL, RDF, and XML) such as OAT (OpenLink Ajax Toolkit). For instance, we've successfully used this to build a Visual Query Building Tool for SPARQL (alpha) that really takes a lot of the pain out of constructing SPARQL Queries (there is much more to come on this front re. handling of DISTINCT, FILTER, ORDER BY etc..).

For now, take a look at the SPARQL Query dump generated by this SIOC & SKOS SPARQL QBE Canvas Screenshot.

You can cut and paste the queries that follow into the Query Builder or use the screenshot to build your variation of this query sample. Alternatively, you can simply click on *This* SPARQL Protocol URL to see the query results in a basic HTML Table. And one last thing, you can grab the SPARQL Query File saved into my ODS-Briefcase (the WebDAV repository aspect of my Data Space).

Note the following SPARQL Protocol Endpoints:

1. MyOpenLink Data Space
2. Experimental Data Space SPARQL Query Builder (you need to register at http://myopenlink.net:8890/ods to use this version)
3. Live Demo Sever
4. Demo Server SPARQL Query Builder (use: demo for both username and pwd when prompted)

My beautified Version of the SPARQL Generated by QBE (you can cut and paste into "Advanced Query" section of QBE) is presented below:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX dct: <http://purl.org/dc/elements/1.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>


SELECT distinct 
       ?forum_name, 
       ?owner, 
       ?post, 
       ?title, 
       ?link, 
       ?url, 
       ?tag
FROM <http://myopenlink.net/dataspace>
WHERE {
       ?forum a sioc:Forum;
                   sioc:type "bookmark";
                   sioc:id ?forum_name;
                   sioc:has_member ?owner.
       ?owner sioc:id "kidehen".
       ?forum sioc:container_of ?post .
       ?post  dct:title ?title .
       optional { ?post sioc:link ?link  }
       optional { ?post sioc:links_to ?url }
       optional { ?post sioc:topic ?topic.
                        ?topic a skos:Concept;
                                  skos:prefLabel ?tag}.
     } 

Unmodified dump from the QBE (this will be beautified automatically in due course by the QBE):

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX dct: <http://purl.org/dc/elements/1.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>


SELECT ?var8 ?var9 ?var13 ?var14 ?var24 ?var27 ?var29 ?var54 ?var56
WHERE
{
graph ?graph {
 ?var8 rdf:type sioc:Forum .
 ?var8 sioc:container_of ?var9 .
 ?var8 sioc:type "bookmark" .
 ?var8 sioc:id ?var54 .
 ?var8 sioc:has_member ?var56 .
 ?var9 rdf:type sioc:Post .
 OPTIONAL {?var9 dc:title ?var13} .
 OPTIONAL {?var9 sioc:links_to ?var14} .
 OPTIONAL {?var9 sioc:link ?var29} .
 ?var9 sioc:has_creator ?var37 .
 OPTIONAL {?var9 sioc:topic ?var24} .
 ?var24 rdf:type skos:Concept .
 OPTIONAL {?var24 skos:prefLabel ?var27} .
 ?var56 rdf:type sioc:User .
 ?var56 sioc:id "kidehen" .
 }
} 

Current missing items re. Visual QBE for SPARQL are:

1. Ability to Save properly to WebDAV so that I can then expose various saved SPARQL Queries (.rq file) from my Data Space via URIs
2. Handling of DISTINCT, FILTERS (note: OPTIONAL is handled via dotted predicate-links)
3. General tidying up re. click event handling etc.

1. Export some bookmarks from your local browser to the usual HTML bookmarks dump file
2. Create an ODS-Bookmarks Instance using your new ODS account
3. Use the ODS-Bookmark Instance to import your local bookmarks from the HTML dump file
4. Repeat the same import sequence using the ODS-Bookmark Instance, but this time pick the del.icio.us option
5. Build your query (change 'kidehen' to your ODS-user-name)
6. That's it you now have Semantic Web presence in the form of a Data Space for your local and del.icio.us hosted bookmarks with tags integrated

Quick Query Builder Tip: You will need to import the following (using the Import Button in the Ontologies & Schemas side-bar);

1. http://www.w3.org/1999/02/22-rdf-syntax-ns# (RDF)
2. http://rdfs.org/sioc/ns# (SIOC)
3. http://purl.org/dc/elements/1.1/ (Dublin Core)
4. http://www.w3.org/2004/02/skos/core# (SKOS)

Browser Support: The SPARQL QBE is SVG based and currently works fine with the following browsers; Firefox 1.5/2.0, Camino (Cocoa variant of Firefox for Mac OS X), Webkit (Safari pre-release / advanced sibling), Opera 9.x. We are evaluating the use of the Adobe SVG plugin re. IE 6/7 support.

Of course this should be a screencast, but I am the middle of a plethora of things right now :-)

I came across a pretty deep comments trail about the aforementioned items on Fred Wilson's blog (aptly titled: A VC) under the subject heading: Web 3.0 Is The Semantic Web.

Contributions to the general Semantic Web discourse by way of responses to valuable questions and commentary contributed by a Semantic Web skeptic (Ed Addison who may be this Ed Addison according to Google):

Ed, Responses to your points re. Semantic Web Matrialization:

<< 1) ontologies can be created and maintained by text extractors and crawlers" >>

Ontologies will be developed by Humans. This process has already commenced and far more landscape has been covered that you may be aware of. For instance, there is an Ontology for Online Communities with Semantics factored in. More importantly, most Blogs, Wikis, and other "points of presence" on the Web are already capable of generating Instance Data for this Ontology by way of the underlying platforms that drive these things. The Ontology is called: SIOC (Semantically-Interlinked Online Communities).

<< 2) the entire web can be marked up, semantically indexed, and maintained by spiders without human assistance >>

Most of it can, and already is :-) Human assistance should, and would, be on an "exception basis" a preferred use of human time (IMHO). We do not need to annotate the Web manually when this labor intensive process can be automated (see my earlier comments).

<< 3) inference over the semantic web does not require an extremely deep heuristic search down multiple, redundant, cyclical pathways with many islands that are disconnected >>

When you have a foundation layer of RDF Data (generated in the manner I've discussed above), you then have a substrate that's far more palatable to Intelligent Reasoning. Note, the Semantic Web is made of many layers. The critical layer at this juncture is the Data-Web (Web of RDF Data). Note, when I refer to RDF I am not referring to RDF/XML the serialization format, I am referring to the Data Model (a Graph).

<< 4) the web becomes smart enough to eliminate websites or data elements that are incorrect, misleading, false, or just plain lousy >>

The Semantic Web vision is not about eliminating Web Sites (The Hypertext-Document-Web). It is simply about adding another dimension of interaction to the Web. This is just like the Services-Web dimension as delivered by Web 2.0.

We are simply evolving within an innovation continuum. There is no mutual exclusivity about any of the Web Dimensions since they collectively provide us with a more powerful infrastructure for building and exploiting "collective wisdom".

As for the Data-Web experiment part of this post, I would expect to see this post exposed as another contribution to the Data-Web via the PingTheSemanticWeb notification service :-) Implying, that all the relevant parts of this conversation are in a format (Instance Data for the SIOC Ontology) that is available for further use in a myriad of forms.

Dimension 1 = Interactive Web (Visual Web of HTML based Sites aka Web 1.0)

Dimension 2 = Services Web (Presence based Web of Services; a usage pattern commonly referred to as Web 2.0)

Dimension 3 = Data Web (Presence and Open Data Access based Web of Databases aka Semantic Web layer 1)

Dimension 4 = Ontology Web (Intelligent Agent palatable Web aka Semantic Web layer 2)

....

Hopefully, I can expand further :-)

For additional clarity re. my comments above, you can also look at the SPARQL & SIOC Usecase samples document for our OpenLink Data Spaces platform. Bottom line, the Semantic Web and SPARQL aren't BORING. In fact, quite the contrary, since they are essential ingredients of a more powerful Web than the one we work with today!

Enjoy the rest of John's post:

Creating connections between discussion clouds with SIOC:

(Extract from our forthcoming BlogTalk paper about browsers for SIOC.)

SIOC provides a unified vocabulary for content and interaction description: a semantic layer that can co-exist with existing discussion platforms. Using SIOC, various linkages are created between the aforementioned concepts, which allow new methods of accessing this linked data, including:

• Virtual Forums. These may be a gathering of posts or threads which are distributed across discussion platforms, for example, where a user has found posts from a number of blogs that can be associated with a particular category of interest, or an agent identifies relevant posts across a certain timeframe.
• Distributed Conversations. Trackbacks are commonly used to link blog posts to previous posts on a related topic. By creating links in both directions, not only across blogs but across all types of internet discussions, conversations can be followed regardless of what point or URI fragment a browser enters at.
• Unified Communities. Apart from creating a web page with a number of relevant links to the blogs or forums or people involved in a particular community, there is no standard way to define what makes up an online community (apart from grouping the people who are members of that community using FOAF or OPML). SIOC allows one to simply define what objects are constituent parts of a community, or to say to what community an object belongs (using sioc:has_part / part_of): users, groups, forums, blogs, etc.
• Shared Topics. Technorati (a search engine for blogs) and BoardTracker (for bulletin boards) have been leveraging the free-text tags that people associate with their posts for some time now. SIOC allows the definition of such tags (using the subject property), but also enables hierarchial or non-hierarchial topic definition of posts using sioc:topic when a topic is ambiguous or more information on a topic is required. Combining with other Semantic Web vocabularies, tags and topics can be further described using the SKOS organisation system.
• One Person, Many User Accounts. SIOC also aims to help the issue of multiple identities by allowing users to define that they hold other accounts or that their accounts belong to a particular personal identity (via foaf:holdsOnlineAccount or sioc:account_of). Therefore, all the posts or comments made by a particular person using their various associated user accounts across platforms could be identified.

A phase in the evolution web usage patterns that emphasizes Web Services based interaction between “Web Users” and “Points of Web Presence” over traditional “Web Users” and “Web Sites” based interaction. Basically, a transition from visual site interaction to presence based interaction.

BTW - Dare Obasanjo also commented about Web usage patterns in his post titled: The Two Webs. Where he concluded that we had a dichotomy along the lines of: HTTP-for-APIs (2.0) and HTTP-for-Browsers (1.0). Which Jon Udell evolved into: HTTP-Services-Web and HTTP-Intereactive-Web during our recent podcast conversation.

With definitions in place, I will resume my quest to unveil the aforementioned Web 2.0 Data Access Conundrum:

• Emphasis on XML's prowess in the realms of Data and Protocol Modeling alongside Data Representation. Especially as SOAP or REST styles of Web Services and various XML formats (RSS 0.92/1.0/1.1/2.0, Atom, OPML, OCS etc.) collectively define the Web 2.0 infrastructure landscape
• Where a modicum of Data Access appreciation and comprehension does exist it is inherently compromised by business models that mandate some form of “Walled Gardens” and “Data Silos”
• Mash-ups are a response to said “Walled Gardens” and “Data Silos” . Mash-ups by definition imply combining things that were not built for recombination.

As you can see from the above, Open Data access isn't genuinely compatible with Web 2.0.

We can also look at the same issue by way of the popular M-V-C (Model View Controller) pattern. Web 2.0 is all about the “V” and “C” with a modicum of “M” at best (data access, open data access, and flexible open data access are completely separate things). The “C” items represent application logic exposed by SOAP or REST style web services etc. I'll return to this later in this post.

What about Social Networking you must be thinking? Isn't this a Web 2.0 manifestation? Not at all (IMHO). The Web was developed / invented by Tim Berners-Lee to leverage the “Network Effects” potential of the Internet for connecting People and Data. Social Networking on the other hand, is simply one of several ways by which construct network connections. I am sure we all accept the fact that connections are built for many other reasons beyond social interaction. That said, we also know that through social interactions we actually develop some of our most valuable relationships (we are social creatures after-all).

The Web 2.0 Open Data Access impedance reality is ultimately going to be the greatest piece of tutorial and usecase material for the Semantic Web. I take this position because it is human nature to seek Freedom (in unadulterated form) which implies the following:

• Access Data from a myriad of data sources (irrespective of structural differences at the database level)
• Mesh (not Mash) data in new and interesting ways
• Share the meshed data with as many relevant people as possible for social, professional, political, religious, and other reasons
• Construct valuable networks based on data oriented connections

Web 2.0 by definition and use case scenarios is inherently incompatible with the above due to the lack of Flexible and Open Data Access.

If we take the definition of Web 2.0 (above) and rework it with an appreciation Flexible and Open Data Access you would arrive at something like this:

A phase in the evolution of the web that emphasizes interaction between “Web Users” and “Web Data” facilitated by Web Services based APIs and an Open & Flexible Data Access Model “.

In more succinct form:

A pervasive network of people connected by data or data connected by people.

Returning to M-V-C and looking at the definition above, you now have a complete of ”M“ which is enigmatic in Web 2.0 and the essence of the Semantic Web (Data and Context).

To make all of this possible a palatable Data Model is required. The model of choice is the Graph based RDF Data Model - not to be mistaken for the RDF/XML serialization which is just that, a data serialization that conforms to the aforementioned RDF data model.

The Enterprise Challenge

Web 2.0 cannot and will not make valuable inroads into the the enterprise because enterprises live and die by their ability to exploit data. Weblogs, Wikis, Shared Bookmarking Systems, and other Web 2.0 distributed collaborative applications profiles are only valuable if the data is available to the enterprise for meshing (not mashing).

A good example of how enterprises will exploit data by leveraging networks of people and data (social networks in this case) is shown in this nice presentation by Accenture's Institute for High Performance Business titled: Visualizing Organizational Change.

Web 2.0 commentators (for the most part) continue to ponder the use of Web 2.0 within the enterprise while forgetting the congruency between enterprise agility and exploitation of people & data networks (The very issue emphasized in this original Web vision document by Tim Berners-Lee). Even worse, they remain challenged or spooked by the Semantic Web vision because they do not understand that Web 2.0 is fundamentally a Semantic Web precursor due to Open Data Access challenges. Web 2.0 is one of the greatest demonstrations of why we need the Semantic Web at the current time.

Finally, juxtapose the items below and you may even get a clearer view of what I am an attempting to convey about the virtues of Open Data Access and the inflective role it plays as we move beyond Web 2.0:

Information Management Proposal - Tim Berners-Lee
Visualizing Organizational Change - Accenture Institute of High Performance Business

A phase in the evolution web usage patterns that emphasizes Web Services based interaction between “Web Users” and “Points of Web Presence” over traditional “Web Users” and “Web Sites” based interaction. Basically, a transition from visual site interaction to presence based interaction.

BTW - Dare Obasanjo also commented about Web usage patterns in his post titled: The Two Webs. Where he concluded that we had a dichotomy along the lines of: HTTP-for-APIs (2.0) and HTTP-for-Browsers (1.0). Which Jon Udell evolved into: HTTP-Services-Web and HTTP-Intereactive-Web during our recent podcast conversation.

With definitions in place, I will resume my quest to unveil the aforementioned Web 2.0 Data Access Conundrum:

• Emphasis on XML's prowess in the realms of Data and Protocol Modeling alongside Data Representation. Especially as SOAP or REST styles of Web Services and various XML formats (RSS 0.92/1.0/1.1/2.0, Atom, OPML, OCS etc.) collectively define the Web 2.0 infrastructure landscape
• Where a modicum of Data Access appreciation and comprehension does exist it is inherently compromised by business models that mandate some form of “Walled Gardens” and “Data Silos”
• Mash-ups are a response to said “Walled Gardens” and “Data Silos” . Mash-ups by definition imply combining things that were not built for recombination.

As you can see from the above, Open Data access isn't genuinely compatible with Web 2.0.

We can also look at the same issue by way of the popular M-V-C (Model View Controller) pattern. Web 2.0 is all about the “V” and “C” with a modicum of “M” at best (data access, open data access, and flexible open data access are completely separate things). The “C” items represent application logic exposed by SOAP or REST style web services etc. I'll return to this later in this post.

What about Social Networking you must be thinking? Isn't this a Web 2.0 manifestation? Not at all (IMHO). The Web was developed / invented by Tim Berners-Lee to leverage the “Network Effects” potential of the Internet for connecting People and Data. Social Networking on the other hand, is simply one of several ways by which construct network connections. I am sure we all accept the fact that connections are built for many other reasons beyond social interaction. That said, we also know that through social interactions we actually develop some of our most valuable relationships (we are social creatures after-all).

The Web 2.0 Open Data Access impedance reality is ultimately going to be the greatest piece of tutorial and usecase material for the Semantic Web. I take this position because it is human nature to seek Freedom (in unadulterated form) which implies the following:

• Access Data from a myriad of data sources (irrespective of structural differences at the database level)
• Mesh (not Mash) data in new and interesting ways
• Share the meshed data with as many relevant people as possible for social, professional, political, religious, and other reasons
• Construct valuable networks based on data oriented connections

Web 2.0 by definition and use case scenarios is inherently incompatible with the above due to the lack of Flexible and Open Data Access.

If we take the definition of Web 2.0 (above) and rework it with an appreciation Flexible and Open Data Access you would arrive at something like this:

A phase in the evolution of the web that emphasizes interaction between “Web Users” and “Web Data” facilitated by Web Services based APIs and an Open & Flexible Data Access Model “.

In more succinct form:

A pervasive network of people connected by data or data connected by people.

Returning to M-V-C and looking at the definition above, you now have a complete of ”M“ which is enigmatic in Web 2.0 and the essence of the Semantic Web (Data and Context).

To make all of this possible a palatable Data Model is required. The model of choice is the Graph based RDF Data Model - not to be mistaken for the RDF/XML serialization which is just that, a data serialization that conforms to the aforementioned RDF data model.

The Enterprise Challenge

Web 2.0 cannot and will not make valuable inroads into the the enterprise because enterprises live and die by their ability to exploit data. Weblogs, Wikis, Shared Bookmarking Systems, and other Web 2.0 distributed collaborative applications profiles are only valuable if the data is available to the enterprise for meshing (not mashing).

A good example of how enterprises will exploit data by leveraging networks of people and data (social networks in this case) is shown in this nice presentation by Accenture's Institute for High Performance Business titled: Visualizing Organizational Change.

Web 2.0 commentators (for the most part) continue to ponder the use of Web 2.0 within the enterprise while forgetting the congruency between enterprise agility and exploitation of people & data networks (The very issue emphasized in this original Web vision document by Tim Berners-Lee). Even worse, they remain challenged or spooked by the Semantic Web vision because they do not understand that Web 2.0 is fundamentally a Semantic Web precursor due to Open Data Access challenges. Web 2.0 is one of the greatest demonstrations of why we need the Semantic Web at the current time.

Finally, juxtapose the items below and you may even get a clearer view of what I am an attempting to convey about the virtues of Open Data Access and the inflective role it plays as we move beyond Web 2.0:

Information Management Proposal - Tim Berners-Lee
Visualizing Organizational Change - Accenture Institute of High Performance Business

Continuing from our recent Podcast conversation, Jon Udell sheds further insight into the essence of our conversation via a “Strategic Developer” column article titled: Accessing the web of databases.

Below, I present an initial dump of a DataSpace FAQ below that hopefully sheds light on the DataSpace vision espoused during my podcast conversation with Jon.

What is a DataSpace?

A moniker for Web-accessible atomic containers that manage and expose Data, Information, Services, Processes, and Knowledge.

What would you typically find in a Data Space? Examples include:

• Raw Data - SQL, HTML, XML (raw), XHTML, RDF etc.
  
  
• Information (Data In Context) - XHTML (various microformats), Blog Posts (in RSS, Atom, RSS-RDF formats), Subscription Lists (OPML, OCS, etc), Social Networks (FOAF, XFN etc.), and many other forms of applied XML.

• Web Services (Application/Service Logic) - REST or SOAP based invocation of application logic for context sensitive and controlled data access and manipulation.

• Persisted Knowledge - Information in actionable context that is also available in transient or persistent forms expressed using a Graph Data Model. A modern knowledgebase would more than likely have RDF as its Data Language, RDFS as its Schema Language, and OWL as its Domain  Definition (Ontology) Language. Actual Domain, Schema, and Instance Data would be serialized using formats such as RDF-XML, N3, Turtle etc).

How do Data Spaces and Databases differ?
Data Spaces are fundamentally problem-domain-specific database applications. They offer functionality that you would instinctively expect of a database (e.g. AICD data management) with the additonal benefit of being data model and query language agnostic. Data Spaces are for the most part DBMS Engine and Data Access Middleware hybrids in the sense that ownership and control of data is inherently loosely-coupled.

How do Data Spaces and Content Management Systems differ?
Data Spaces are inherently more flexible, they support multiple data models and data representation formats. Content management systems do not possess the same degree of data model and data representation dexterity.

How do Data Spaces and Knowledgebases differ?
A Data Space cannot dictate the perception of its content. For instance, what I may consider as knowledge relative to my Data Space may not be the case to a remote client that interacts with it from a distance, Thus, defining my Data Space as Knowledgebase, purely, introduces constraints that reduce its broader effectiveness to third party clients (applications, services, users etc..). A Knowledgebase is based on a Graph Data Model resulting in significant impedance for clients that are built around alternative models. To reiterate, Data Spaces support multiple data models.

What Architectural Components make up a Data Space?

• ORDBMS Engine - for Data Modeling agility (via complex purpose specific data types and data access methods), Data Atomicity, Data Concurrency, Transaction Isolation, and Durability (aka ACID).
  
  
• Virtual Database Engine - for creating a single view of, and access point to, heterogeneous SQL, XML, Free Text, and other data. This is all about Virtualization at the Data Access Level.

• Web Services Platform - enabling controlled access and manipulation (via application, service, or protocol logic) of Virtualized or Disparate Data. This layer handles the decoupling of functionality from monolithic wholes for function specific invocation via Web Services using either the SOAP or REST approach.

Where can I see a DataSpace along the lines described, in action?

Just look at my blog, and take the journey as follows:

• Front Door (Web 1.0)
• Lounge (Web 2.0) via GData or OpenSearch
• Floor Plan via FOAF or SIOC RDF Data Sets (Graphs)
• Rest of the house (beyond Web 2.0) sending  SPARQL Queries to a SPARQL Endpoint.
  

What about other Data Spaces?

There are several and I will attempt to categorize along the lines of query method available:
Type 1 (Free Text Search over HTTP):
Google, MSN, Yahoo!, Amazon, eBay, and most Web 2.0 plays .

Type 2 (Free Text Search and XQuery/XPath over HTTP)
A few blogs and Wikis (Jon Udell's and a few others)

•    SIOC enabled sites (aka points of semantic web presence)
  
•    PingTheSemantic
  

• My Blog Data Space (as stated earlier in this post)
  
• My General Data Space - (ditto; note that this is currently experimental)
  

What About Data Space aware tools?

•    OpenLink Ajax Toolkit - provides Javascript Control level binding to Query Services such as XMLA for SQL, GData for Free Text, OpenSearch for Free Text, SPARQL for RDF, in addition to service specific Web Services (Web 2.0 hosted solutions that expose service specific APIs)
•    Semantic Radar - a Firefox Extension
•    PingTheSemantic - the Semantic Webs equivalent of Web 2.0's weblogs.com
•    PiggyBank - a Firefox Extension

I was compelled to go back to the RSS 2.0 imbroglio when I came across Dave Winer's comments re. "the SEC attempting to reinvent RSS 2.0..." response to Jon Udell's recent XBRL article.

Although I don't believe in complex entry points into complex technology realms, I do subscribe to the approach where developers deal with the complexity associated with a problem domain while hiding said complexity from ambivalent end-users via coherent interfaces -- which does not always imply User Interface.

XBRL is a great piece of work that addresses the complex problem domain of Financial Reporting. The only thing it's missing right now is an Ontology that facilitates RDF Data Model based XBRL Schema and Instance Data which ultimately makes XBRL data available to RDF query languages such as SPARQL. This line of thought implies, for instance, an XML Schema to OWL Ontology Mapping for Schema Data (as explained in a white paper by the VSIS Group at the university of Hamburg) leaving the Instance Data to be generated in a myriad of ways that includes XML to RDF and/or XML->SQL->RDF.

As I stated in an earlier post: we should not mistake ambivalence to lack of intelligence. Assuming "Simple" is always right at all times is another way of subscribing to this profound misconception. You know, assuming the world was flat (as opposed to geoid) was quite palatable at some point in the history of mankind, I wonder what would have happened if we held on to this point of view to this day because of its "Simplicity"?

OAT offers a broad Javascript-based, browser-independent widget set
for building data source independent rich internet applications that are usable across a broad range of Ajax-capable web browsers.

OAT's support binding to the following data sources via its Ajax Database Connectivity Layer:

SQL Data via XML for Analysis (XMLA)
Web Data via SPARQL, GData, and OpenSearch Query Services
Web Services specific Data via service specific binding to SOAP and REST style web services

The toolkit includes a collection of powerful rich internet application prototypes include: SQL Query By Example, Visual Database Modeling, and Data bound Web Form Designer.

Project homepage on sourceforge.net:

http://sourceforge.net/projects/oat

Source Code:

http://sourceforge.net/projects/oat/files

Live demonstration:

http://www.openlinksw.com/oat/

(Via Valentin Zacharias.)

GeoRSS & Geonames for Philanthropy: "

I heard about Kiva.ORG in a BusinessWeek podcast. After visiting its website, I think there are few places where GeoRSS (in the RDF/A syntax) and Geonames can be used to enhance the site’s functionality.

Kiva.ORG Background

It’s a microfinance website for people in the developing countries. Its business model is in the intersection between peer-to-peer financing and philanthropy. The goal is to help developing country businesses to borrow small loans from a large group of Web users, so that they can avoid paying high interests to the banks.

For example, a person in Uganda can request a $500 loan and use it for buying and selling more poultry. One or more lenders (anyone on the Web) may decide to grant loans to that person in increments as tiny as $25. After few years, that person will pay back the loans to the lenders.

How GeoRSS and Geonames Can Help

I went to the website and discovered the site has a relative weak search and browsing interface. In particular, there is no way to group loan requests based on geographical locations (e.g., countries, cities and regions).

Took a look at individual loan pages. Each page actually has standard ways to describe location information — e.g., Location: Mbale, Uganda.

It should be relative easy to add GeoRSS points (in the RDF/A syntax) to describe these location information (an alternative maybe using Microformat Geo or W3C Geo). Once the location information is annotated, one can imagine building a map mashup to display loan requests in a geospatial perspective. One can also build search engines to support spatial queries such as ‘find me all loans with from Mbale’.

Since Kiva.ORG webmasters may not be GIS experts, it will be nice if we can find ways to automatically geocode location information and describe that using GeoRSS. This automatic geocoding procedure can be developed using Geonames’s webservices. Take a string ‘Mbale’ or ‘Uganda’, and send to Geonames’s search service. The procedure will get back JSON or XML description of the location, which include latitude and longitude. This will then be used to annotate the location information in a Kiva loan page.

Can you think of other ways to help Kiva.ORG to become more ‘geospatially intelligent’?
You can learn more about Kiva.ORG at its website and listen to this podcast.

Standards as social contracts: "Looking at Dave Winer's efforts in evangelizing OPML, I try to draw some rough lines into what makes a de-facto standard. De Facto standards are made and seldom happen on their own. In this entry, I look back at the history of HTML, RSS, the open source movement and try to draw some lines as to what makes a standard.

(Via Tristan Louis.)

I posted a comment to the Tristan Louis' post along the following lines:

Analysis is spot on re. the link between de facto standardization and bootstrapping. Likewise, the clear linkage between boostrapping and connected communities (a variation of the social networking paradigm).

Dave built a community around a XML content syndication and subscription usecase demo that we know today as the blogosphere. Superficially, one may conclude that Semantic Web vision has suffered to date from a lack a similar bootstrap effort. Whereas in reality, we are dealing with "time and context" issues that are critical to the base understanding upon which a "Dave Winer" style bootstrap for the Semantic Web would occur.

Personally, I see the emergence of Web 2.0 (esp. the mashups phenomenon) as the "time and context" seeds from which the Semantic Web bootstrap will sprout. I see shared ontologies such as FOAF and SIOC leading the way (they are the RSS 2.0's of the Semantic Web IMHO).

"We all know that structured data is boring and useless; while unstructured data is sexy and chock full of value. Well, only up to a point, Lord Copper. Genuinely unstructured data can be a real nuisance - imagine extracting the return address from an unstructured letter, without letterhead and any of the formatting usually applied to letters. A letter may be thought of as unstructured data, but most business letters are, in fact, highly-structured." ....

"The labels "structured data" and "unstructured data" are often used ambiguously by different interest groups; and often used lazily to cover multiple distinct aspects of the issue. In reality, there are at least three orthogonal aspects to structure:

* The structure of the data itself.

* The structure of the container that hosts the data.

* The structure of the access method used to access the data.

These three dimensions are largely independent and one does not need to imply another. For example, it is absolutely feasible and reasonable to store unstructured data in a structured database container and access it by unstructured search mechanisms."

Data understanding and appreciation is dwindling at a time when the reverse should be happening. We are supposed to be in the throws of the "Information Age", but for some reason this appears to have no correlation with data and "data access" in the minds of many -- as reflected in the broad contradictory positions taken re. unstructured data vs structured data, structured is boring and useless while unstructured is useful and sexy....

The difference between "Structured Containers" and "Structured Data" are clearly misunderstood by most (an unfortunate fact).

For instance all DBMS products are "Structured Containers" aligned to one or more data models (typically one). These products have been limited by proprietary data access APIs and underlying data model specificity when used in the "Open-world" model that is at the core of the World Wide Web. This confusion also carries over to the misconception that Web 2.0 and the Semantic/Data Web are mutually exclusive.

But things are changing fast, and the concept of multi-model DBMS products is beginning to crystalize. On our part, we have finally released the long promised "OpenLink Data Spaces" application layer that has been developed using our Virtuoso Universal Server. We have structured unified storage containment exposed to the data web cloud via endpoints for querying or accessing data using a variety of mechanisms that include; GData, OpenSearch, SPARQL, XQuery/XPath, SQL etc..

To be continued....

The screencasts covered the following functionality realms:

1. SQL Query By Example (basic)
2. SQL Query By Example (advanced - pivot table construction)
3. Web Form Design (basic database driven map based mashup)
4. Web Form Design (advanced database driven map based mashup)

To bring additional clarity to the screencasts demos and OAT in general, I have saved a number of documents that are the by products of activities in the screenvcasts:

1. Live XML Document produced using SQL Query By Example (basic) (you can use drag and drop columns across the grid to reorder and sort presentation)
2. Live XML Document produced using QBE and Pivot Functionality (you can drag and drop the aggregate columns and rows to create your own views etc..)
3. Basic database driven map based mashup (works with FireFox, Webkit, Camino; click on pins to see national flag)
4. Advanced database driven map based mashup (works with FireFox, Webkit, Camino; records, 36, 87, and 257 will unveil pivots via lookup pin)

Notes:

• “Advanced”, as used above, simply means that I am embedding images (employee photos and national flags) and a database driven pivot into the map pins that serve as details lookups in classic SQL master/details type scenarios.
• The “Ajax Call In Progress..” dialog is there to show live interaction with a remote database (in this case Virtuoso but this could be any ODBC, JDBC, OLEDB, ADO.NET, or XMLA accessible data source)
• The data access magic source (if you want to call it that) is XMLA - a standard that has been in place for years but completely misunderstood and as a result under utilized

You can see a full collection of saved documents at the following locations:

• My Mashups demo directory (Google and Yahoo! demo variants but note these do not work with Safari or IE at the current time. IE7 issues will be resolved in the next day or so)
• My Pivots demo directory (other Pivots will be added as I build and save them)
• My Saved Queries (a collection of saved QBE generated queries)

Two graphs that explain most IT dysfunction (Part I): "

Inspired by reading about other people’s blogging weaknesses, I’ve decided to finally get this one off the back burner and post it. I’m pretty sure that this isn’t original, but I started thinking about this way back in 1996 (pre-social-bookmarking) and I’ve lost my pointer to whatever influenced it. Anybody who can set me straight- I’d appreciate it.

So here goes.

There are two graphs which, when seen together, explain a hell of a lot about various forms of dysfunction that you see in the technology world.

In this first graph, X represents relative ‘technical expertise’ and Y represents the ‘perceived benefit’ in the introduction of a new technology:

The summary is that technical neophytes (A) tend to see high potential benefit in new technologies, while people who have a bit of technology experience (B) grow increasingly cynical about technology claims and can rattle-off the names of technologies that they have seen over-hyped and that have under-delivered. The interesting thing though, is that, as people become really expert in technology (C), their view of the potential benefits in new technology starts to increase again. At the far right of this scale I’m talking about the real experts- the alpha-geeks of the world.

In the second graph, X again represents technical expertise, but Y represents ‘perceived risk’ associated with the introduction of a new technology:

Here the curve is inverted, but the basic pattern is the same. The neophytes (A) are blissfully unaware of the things that can go wrong with the introduction of a new technology. The tech-savvy (B) are battle-scarred and have seen (and possibly caused) countless disasters. The alpha-geeks (C) have also seen their share of problems, but they have also learned from their mistakes and know how to avoid them in the future. The alpha-geeks understand how to manage the risk.

Now things get interesting when you map these two dynamics against each other:

You see that neophytes in group A have essentially the same world view as the alpha-geeks in group C, but for completely different reasons. The trouble starts when you realize that most of senior executives, venture capitalists and members of the popular press are in group A. At the other extreme, most R&D groups, architecture groups, independent consultancies, technology pundits, etc. are in group C . There are a few problems with this:

• People in group A will often talk to and solicit advice from people in group C
• There are relatively few people in group C
• Most of the people who actually have to implement new technologies are in group B.

So you can start to see the problem.

In Part II I’l talk some more about group B and I’ll discuss some of the classic patterns that emerge when A, B and C try to work with each other.

"

I added the missing piece regarding the "Virtuoso Conductor" (the Web based Admin UI for Virtuoso) to the original post below. I also added a link to our live SPARQL Demo so that anyone interested can start playing around with SPARQL and SPARQL integrated into SQL right away.

Another good thing about this post is the vast amount of valuable links that it contains. To really appreciate this point simply visit my Linkblog (excuse the current layout :-) - a Tab if you come in via the front door of this Data Space (what I used to call My Weblog Home Page).

"Free" Databases: Express vs. Open-Source RDBMSs: "Open-source relational database management systems (RDBMSs) are gaining IT mindshare at a rapid pace. As an example, BusinessWeek's February 6, 2006 ' Taking On the Database Giants ' article asks 'Can open-source upstarts compete with Oracle, IBM, and Microsoft?' and then provides the answer: 'It's an uphill battle, but customers are starting to look at the alternatives.'

There's no shortage of open-source alternatives to look at. The BusinessWeek article concentrates on MySQL, which BW says 'is trying to be the Ikea of the database world: cheap, needs some assembly, but has a sleek, modern design and does the job.' The article also discusses Postgre[SQL] and Ingres, as well as EnterpriseDB, an Oracle clone created from PostgreSQL code*. Sun includes PostgreSQL with Solaris 10 and, as of April 6, 2006, with Solaris Express.**

*Frank Batten, Jr., the investor who originally funded Red Hat, invested a reported $16 million into Great Bridge with the hope of making a business out of providing paid support to PostgreSQL users. Great Bridge stayed in business only 18 months , having missed an opportunity to sell the business to Red Hat and finding that selling $50,000-per-year support packages for an open-source database wasn't easy. As Batten concluded, 'We could not get customers to pay us big dollars for support contracts.' Perhaps EnterpriseDB will be more successful with a choice of $5,000, $3,000, or $1,000 annual support subscriptions .

**Interestingly, Oracle announced in November 2005 that Solaris 10 is 'its preferred development and deployment platform for most x64 architectures, including x64 (x86, 64-bit) AMD Opteron and Intel Xeon processor-based systems and Sun's UltraSPARC(R)-based systems.'

There is a surfeit of reviews of current MySQL, PostgreSQL and—to a lesser extent—Ingres implementations. These three open-source RDBMSs come with their own or third-party management tools. These systems compete against free versions of commercial (proprietary) databases: SQL Server 2005 Express Edition (and its MSDE 2000 and 1.0 predecessors), Oracle Database 10g Express Edition, IBM DB2 Express-C, and Sybase ASE Express Edition for Linux where database size and processor count limitations aren't important. Click here for a summary of recent InfoWorld reviews of the full versions of these four databases plus MySQL, which should be valid for Express editions also. The FTPOnline Special Report article, 'Microsoft SQL Server Turns 17,' that contains the preceding table is here (requires registration.)

SQL Server 2005 Express Edition SP-1 Advanced Features

SQL Server 2005 Express Edition with Advanced Features enhances SQL Server 2005 Express Edition (SQL Express or SSX) dramatically, so it deserves special treatment here. SQL Express gains full text indexing and now supports SQL Server Reporting Services (SSRS) on the local SSX instance. The SP-1 with Advanced Features setup package, which Microsoft released on April 18, 2006, installs the release version of SQL Server Management Studio Express (SSMSE) and the full version of Business Intelligence Development Studio (BIDS) for designing and editing SSRS reports. My 'Install SP-1 for SQL Server 2005 and Express' article for FTPOnline's SQL Server Special Report provides detailed, illustrated installation instructions for and related information about the release version of SP-1. SP-1 makes SSX the most capable of all currently available Express editions of commercial RDBMSs for Windows.

OpenLink Software's Virtuoso Open-Source Edition

OpenLink Software announced an open-source version of it's Virtuoso Universal Server commercial DBMS on April 11, 2006. On the initial date of this post, May 2, 2006, Virtuoso Open-Source Edition (VOS) was virtually under the radar as an open-source product. According to this press release, the new edition includes:

• SPARQL compliant RDF Triple Store
• SQL-200n Object-Relational Database Engine (SQL, XML, and Free Text)
• Integrated BPEL Server and Enterprise Service Bus
• WebDAV and Native File Server
• Web Application Server that supports PHP, Perl, Python, ASP.NET, JSP, etc.
• Runtime Hosting for Microsoft .NET, Mono, and Java

VOS only lacks the virtual server and replication features that are offered by the commercial edition. VOS includes a Web-based administration tool called the "Virtuoso Conductor" According to Kingsley Idehen's Weblog, 'The Virtuoso build scripts have been successfully tested on Mac OS X (Universal Binary Target), Linux, FreeBSD, and Solaris (AIX, HP-UX, and True64 UNIX will follow soon). A Windows Visual Studio project file is also in the works (ETA some time this week).'

InfoWorld's Jon Udell has tracked Virtuoso's progress since 2002, with an additional article in 2003 and a one-hour podcast with Kingsley Idehen on April 26, 2006. A major talking point for Virtuoso is its support for Atom 0.3 syndication and publication, Atom 1.0 syndication and (forthcoming) publication, and future support for Google's GData protocol, as mentioned in this Idehen post. Yahoo!'s Jeremy Zawodny points out that the 'fingerprints' of Adam Bosworth, Google's VP of Engineering and the primary force behind the development of Microsoft Access, 'are all over GData.' Click here to display a list of all OakLeaf posts that mention Adam Bosworth.

One application for the GData protocol is querying and updating the Google Base database independently of the Google Web client, as mentioned by Jeremy: 'It's not about building an easier onramp to Google Base. ... Well, it is. But, again, that's the small stuff.' Click here for a list of posts about my experiences with Google Base. Watch for a future OakLeaf post on the subject as the GData APIs gain ground.

Open-Source and Free Embedded Database Contenders

Open-source and free embedded SQL databases are gaining importance as the number and types of mobile devices and OSs proliferate. Embedded databases usually consist of Java classes or Windows DLLs that are designed to minimize file size and memory consumption. Embedded databases avoid the installation hassles, heavy resource usage and maintenance cost associated with client/server RDBMSs that run as an operating system service.

Andrew Hudson's December 2005 'Open Source databases rounded up and rodeoed' review for The Enquirer provides brief descriptions of one commercial and eight open source database purveyors/products: Sleepycat, MySQL, PostgreSQL, Ingres, InnoBase, Firebird, IBM Cloudscape (a.k.a, Derby), Genezzo, and Oracle. Oracle Sleepycat* isn't an SQL Database, Oracle InnoDB* is an OEM database engine that's used by MySQL, and Genezzo is a multi-user, multi-server distributed database engine written in Perl. These special-purpose databases are beyond the scope of this post.

* Oracle purchased Sleepycat Software, Inc. in February 2006 and purchased Innobase OY in October 2005 . The press release states: 'Oracle intends to continue developing the InnoDB technology and expand our commitment to open source software.'

Derby is an open-source release by the Apache Software Foundation of the Cloudscape Java-based database that IBM acquired when it bought Informix in 2001. IBM offers a commercial release of Derby as IBM Cloudscape 10.1. Derby is a Java class library that has a relatively light footprint (2 MB), which make it suitable for client/server synchronization with the IBM DB2 Everyplace Sync Server in mobile applications. The IBM DB2 Everyplace Express Edition isn't open source or free*, so it doesn't qualify for this post. The same is true for the corresponding Sybase SQL Anywhere components.**

* IBM DB2 Everyplace Express Edition with synchronization costs $379 per server (up to two processors) and $79 per user. DB2 Everyplace Database Edition (without DB2 synchronization) is $49 per user. (Prices are based on those when IBM announced version 8 in November 2003.)

** Sybase's iAnywhere subsidiary calls SQL Anywhere 'the industry's leading mobile database.' A Sybase SQL Anywhere Personal DB seat license with synchronization to SQL Anywhere Server is $119; the cost without synchronization wasn't available from the Sybase Web site. Sybase SQL Anywhere and IBM DB2 Everyplace perform similar replication functions.

Sun's Java DB, another commercial version of Derby, comes with the Solaris Enterprise Edition, which bundles Solaris 10, the Java Enterprise System, developer tools, desktop infrastructure and N1 management software. A recent Between the Lines blog entry by ZDNet's David Berlind waxes enthusiastic over the use of Java DB embedded in a browser to provide offline persistence. RedMonk analyst James Governor and eWeek's Lisa Vaas wrote about the use of Java DB as a local data store when Tim Bray announced Sun's Derby derivative and Francois Orsini demonstrated Java DB embedded in the Firefox browser at the ApacheCon 2005 conference.

Firebird is derived from Borland's InterBase 6.0 code, the first commercial relational database management system (RDBMS) to be released as open source. Firebird has excellent support for SQL-92 and comes in three versions: Classic, SuperServer and Embedded for Windows, Linux, Solaris, HP-UX, FreeBSD and MacOS X. The embedded version has a 1.4-MB footprint. Release Candidate 1 for Firebird 2.0 became available on March 30, 2006 and is a major improvement over earlier versions. Borland continues to promote InterBase, now at version 7.5, as a small-footprint, embedded database with commercial Server and Client licenses.

SQLite is a featherweight C library for an embedded database that implements most SQL-92 entry- and transitional-level requirements (some through the JDBC driver) and supports transactions within a tiny 250-KB code footprint. Wrappers support a multitude of languages and operating systems, including Windows CE, SmartPhone, Windows Mobile, and Win32. SQLite's primary SQL-92 limitations are lack of nested transactions, inability to alter a table design once committed (other than with RENAME TABLE and ADD COLUMN operations), and foreign-key constraints. SQLite provides read-only views, triggers, and 256-bit encryption of database files. A downside is the the entire database file is locked when while a transaction is in progress. SQLite uses file access permissions in lieu of GRANT and REVOKE commands. Using SQLite involves no license; its code is entirely in the public domain.

The Mozilla Foundation's Unified Storage wiki says this about SQLite: 'SQLite will be the back end for the unified store [for Firefox]. Because it implements a SQL engine, we get querying 'for free', without having to invent our own query language or query execution system. Its code-size footprint is moderate (250k), but it will hopefully simplify much existing code so that the net code-size change should be smaller. It has exceptional performance, and supports concurrent access to the database. Finally, it is released into the public domain, meaning that we will have no licensing issues.'

Vieka Technology, Inc.'s eSQL 2.11 is a port of SQLite to Windows Mobile (Pocket PC and Smartphone) and Win32, and includes development tools for Windows devices and PCs, as well as a .NET native data provider. A conventional ODBC driver also is available. eSQL for Windows (Win32) is free for personal and commercial use; eSQL for Windows Mobile requires a license for commercial (for-profit or business) use.

HSQLDB isn't on most reviewers' radar, which is surprising because it's the default database for OpenOffice.org (OOo) 2.0's Base suite member. HSQLDB 1.8.0.1 is an open-source (BSD license) Java dembedded database engine based on Thomas Mueller's original Hypersonic SQL Project. Using OOo's Base feature requires installing the Java 2.0 Runtime Engine (which is not open-source) or the presence of an alternative open-source engine, such as Kaffe. My prior posts about OOo Base and HSQLDB are here, here and here.

The HSQLDB 1.8.0 documentation on SourceForge states the following regarding SQL-92 and later conformance:

HSQLDB 1.8.0 supports the dialect of SQL defined by SQL standards 92, 99 and 2003. This means where a feature of the standard is supported, e.g. left outer join, the syntax is that specified by the standard text. Many features of SQL92 and 99 up to Advanced Level are supported and here is support for most of SQL 2003 Foundation and several optional features of this standard. However, certain features of the Standards are not supported so no claim is made for full support of any level of the standards.

Other less well-known embedded databases designed for or suited to mobile deployment are Mimer SQL Mobile and VistaDB 2.1 . Neither product is open-source and require paid licensing; VistaDB requires a small up-front payment by developers but offers royalty-free distribution.

Java DB, Firebird embedded, SQLite and eSQL 2.11 are contenders for lightweight PC and mobile device database projects that aren't Windows-only.

SQL Server 2005 Everywhere

If you're a Windows developer, SQL Server Mobile is the logical embedded database choice for mobile applications for Pocket PCs and Smartphones. Microsoft's April 19, 2006 press release delivered the news that SQL Server 2005 Mobile Editon (SQL Mobile or SSM) would gain a big brother—SQL Server 2005 Everywhere Edition.

Currently, the SSM client is licensed (at no charge) to run in production on devices with Windows CE 5.0, Windows Mobile 2003 for Pocket PC or Windows Mobile 5.0, or on PCs with Windows XP Tablet Edition only. SSM also is licensed for development purposes on PCs running Visual Studio 2005. Smart Device replication with SQL Server 2000 SP3 and later databases has been the most common application so far for SSM.

By the end of 2006, Microsoft will license SSE for use on all PCs running any Win32 version or the preceding device OSs. A version of SQL Server Management Studio Express (SSMSE)—updated to support SSE—is expected to release by the end of the year. These features will qualify SSE as the universal embedded database for Windows client and smart-device applications.

For more details on SSE, read John Galloway's April 11, 2006 blog post and my 'SQL Server 2005 Mobile Goes Everywhere' article for the FTPOnline Special Report on SQL Server."

(Via OakLeaf Systems.)

"In my fourth Friday podcast we hear from Kingsley Idehen, CEO of OpenLink Software. I wrote about OpenLink's universal database and app server, Virtuoso, back in 2002 and 2003. Earlier this month Virtuoso became the first mature SQL/XML hybrid to make the transition to open source. The latest incarnation of the product also adds SPARQL (a semantic web query language) to its repertoire. ..."

(Via Jon's Radio.)

BTW - the GData Protocol and Atom 1.0 publishing support will be delivered in both the Open Source and Commercial Edition updates to Virtuoso next week (very little work due to what's already in place).

I make the clarification above to eliminate the possibility of assuming mutual exclusivity of my perspective/vison and Adam's (Jon also makes this important point when he speaks about our opinions being on either side of a spectrum/continuum). I simply want to broaden the scope of this discussion. I am a profound believer in the Semantic Web / Data Web vision, and I predict that we will be querying the Googlebase via SPARQL in the not to distant future (this doesn't mean that netizens will be forced to master SPARQL, absolutely not! But there will be conduit technologies that deal with matter).

Side note: I actually last spoke with Adam at the NY Hilton in 2000 (the day I unveiled Virtuoso to the public for the first time, in person). We bumped into each other and I told him about Virtuoso (at the time the big emphasis was SQL to XML and the vocabulary we had chosen re. SQL extension...), and he told me about his departure from Microsoft and the commencement of his new venture (CrossGain prior to his stint at BEA), what struck me even more was his interest in Linux and Open Source (bearing in mind this was about 3 or so week after he departed Microsoft.)

If you are encountering Virtuoso for the first time via this post or Jon's, please make time to read the product history article on the Virtuoso Wiki (which is one of many Virtuoso based applications that make up our soon to be released OpenLink DataSpace offering).

That said, I better go listen to the podcast :-)

"In my fourth Friday podcast we hear from Kingsley Idehen, CEO of OpenLink Software. I wrote about OpenLink's universal database and app server, Virtuoso, back in 2002 and 2003. Earlier this month Virtuoso became the first mature SQL/XML hybrid to make the transition to open source. The latest incarnation of the product also adds SPARQL (a semantic web query language) to its repertoire. ..."

(Via Jon's Radio.)

BTW - the GData Protocol and Atom 1.0 publishing support will be delivered in both the Open Source and Commercial Edition updates to Virtuoso next week (very little work due to what's already in place).

I make the clarification above to eliminate the possibility of assuming mutual exclusivity of my perspective/vison and Adam's (Jon also makes this important point when he speaks about our opinions being on either side of a spectrum/continuum). I simply want to broaden the scope of this discussion. I am a profound believer in the Semantic Web / Data Web vision, and I predict that we will be querying the Googlebase via SPARQL in the not to distant future (this doesn't mean that netizens will be forced to master SPARQL, absolutely not! But there will be conduit technologies that deal with matter).

Side note: I actually last spoke with Adam at the NY Hilton in 2000 (the day I unveiled Virtuoso to the public for the first time, in person). We bumped into each other and I told him about Virtuoso (at the time the big emphasis was SQL to XML and the vocabulary we had chosen re. SQL extension...), and he told me about his departure from Microsoft and the commencement of his new venture (CrossGain prior to his stint at BEA), what struck me even more was his interest in Linux and Open Source (bearing in mind this was about 3 or so week after he departed Microsoft.)

If you are encountering Virtuoso for the first time via this post or Jon's, please make time to read the product history article on the Virtuoso Wiki (which is one of many Virtuoso based applications that make up our soon to be released OpenLink DataSpace offering).

That said, I better go listen to the podcast :-)

(Via Moreover - XML and metadata news.)

What Is Virtuoso?

A powerful next generation server product that implements otherwise distinct server functionality within a single server product. Think of Virtuoso as the server software analog of a dual core processor where each core represents a traditional server functionality realm.

Where did it come from?

The Virtuoso History page tells the whole story.

What Functionality Does It Provide?

1. Object-Relational DBMS Engine (ORDBMS like PostgreSQL and DBMS engine like MySQL)

2. XML Data Management (with support for XQuery, XPath, XSLT, and XML Schema)

3. RDF Triple Store (or Database) that supports SPARQL (Query Language, Transport Protocol, and XML Results Serialization format)

4. Service Oriented Architecture (it combines a BPEL Engine with an ESB)

5. Web Application Server (supports HTTP/WebDAV)

6. NNTP compliant Discussion Server

90% of the aforementioned functionality has been available in Virtuoso since 2000 with the RDF Triple Store being the only 2006 item.

What Platforms are Supported

The Virtuoso build scripts have been successfully tested on Mac OS X (Universal Binary Target), Linux, FreeBSD, and Solaris (AIX, HP-UX, and True64 UNIX will follow soon). A Windows Visual Studio project file is also in the works (ETA some time this week).

Why Open Source?

Simple, there is no value in a product of this magnitude remaining the "best kept secret". That status works well for our competitors, but absolutely works against the legions of new generation developers, systems integrators, and knowledge workers that need to be aware of what is actually achievable today with the right server architecture.

What Open Source License is it under?

GPL version 2.

What's the business model?

Dual licensing.

The Open Source version of Virtuoso includes all of the functionality listed above. While the Virtual Database (distributed heterogeneous join engine) and Replication Engine (across heterogeneous data sources) functionality will only be available in the commercial version.

Where is the Project Hosted?

On SourceForge.

Is there a product Blog?

Of course!

Up until this point, the Virtuoso Product Blog has been a covert live demonstration of some aspects of Virtuoso (Content Management). My Personal Blog and the Virtuoso Product Blog are actual Virtuoso instances, and have been so since I started blogging in 2003.

Is There a product Wiki?

Sure! The Virtuoso Product Wiki is also an instance of Virtuoso demonstrating another aspect of the Content Management prowess of Virtuoso.

What About Online Documentation?

Yep! Virtuoso Online Documentation is hosted via yet another Virtuoso instance. This particular instance also attempts to demonstrate Free Text search combined with the ability to repurpose well formed content in a myriad of forms (Atom, RSS, RDF, OPML, and OCS).

What about Tutorials and Demos?

The Virtuoso Online Tutorial Site has operated as a live demonstration and tutorial portal for a numbers of years. During the same timeframe (circa. 2001) we also assembled a few Screencast style demos (their look feel certainly show their age; updates are in the works).

BTW - We have also updated the Virtuoso FAQ and also released a number of missing Virtuoso White Papers (amongst many long overdue action items).

My too-long absence from writing much here can be ascribed to two, differently pleasant, activities. First, a fantastic vacation in Cuba, and second, the redesign and launch of the XTech web site.

Of the first, come to my place for dinner and I'll bore you at length about how amazing it was. Of the second, I'd like to bore you right now!

Thanks to Ruby on Rails and a few late nights, the XTech site now has these new features:

• Full conference schedule (apart from 6 Mozilla talks I'm still nailing down)
• A blog. With go-faster Atom 1.0 stripes and everything!
• Details on the newly-added Ajax Developers' Day.

A few more details on the Ajax Developers' Day. As I mentioned before, when putting together the schedule we felt there was a lot of excellent content still missed out (I'm still feeling guilty at having rejected proposals from many good friends and excellent speakers). So, we put together an extra day at the beginning of the conference where we could go further into detail on Ajax technologies.

This day, featuring speakers such as Simon Willison from Yahoo!, XML expert Kurt Cagle and OpenLaszlo's Max Carlson, will allow those working on Ajax projects--either deployment or toolkits--to meet, discuss best practice and move forward on new ideas. Although it's a day-long event, we didn't want to make the price tag as high as a full-day tutorial, so you can register for the cost of a half-day tutorial.

A few implementation details

If that all sounded a little like advertising, here are some technical details worth sharing. The site's CMS is built on Ruby on Rails. Development was done on Linux, with the help of WINE to check out the view from Internet Explorer. The newsletter is managed by the absurdly wonderful CampaignMonitor.

This conference not for sale

Before I went on vacation, there was some debate in various quarters about paid-for plenary and keynote slots in conferences. Though I hope it is obvious, I wanted to state where I, and thus the XTech conference, stand on this issue.

It has always been my policy to maintain a strict separation between the commercial and editorial aspects of XTech. Although each year there's always a company who thinks they can buy a speaking slot, I never let this happen. The content of the conference is formed by editorial selection by the programme committee, who take the scores from the peer review panel as their primary guide.

Aside from what I hope shows in the excellent quality of the talks and generally interesting keynotes (yes, we get it wrong occasionally!), there are two effects on the conference.

• Sponsors are that much more respected. When a sponsor respects the delegates' time and intelligence, but still attends, you know they're serious about engagement with attendees.
  
• A higher portion of the conference cost is in the registration fees than for some other conferences. We're still trying to keep the costs as low as we can, but we're not prepared to compromise the quality of the schedule by letting vendors buy talk time.

(Via Edd Dumbill's Weblog: Behind the Times.)

Techniques for presenting and managing syndication XML (feeds) are disclosed. In one embodiment, a user can modify how a feed is displayed, such as which content (and how much) is displayed, in what order, and how it is formatted. In another embodiment, a modification regarding how a feed is displayed is stored so that it can be used again at a later time. In yet another embodiment, a user can create a custom feed through aggregation and/or filtering of existing feeds. Aggregation includes, for example, merging the articles of multiple feeds to form a new feed. Filtering includes, for example, selecting a subset of articles of a feed based on whether they satisfy a search query. In yet another embodiment, a user can find articles by entering a search query into a search engine that searches feeds, which will identify one or more articles that satisfy the query.

Clearly Apple don't seem to understand the world of XML, so let me give them a quick recap:

1. XML enables separation of Data and Formating

2. It facilitates Data Representation, Transformation (XSLT), Exchange (syndication and subscription), and Modeling (languages, protocols, data models, amongst other things)

3. It is inherently open

4. You can't patent its essence through the back door!

The Blogosphere is a Galaxy within Cyberspace comprised of Solar systems of Blogs that revolve around X-list bloggers, Topics, or more recently Tags; through the gravitational pull of links to RSS (today), Atom (in due course), and RDF (the future).

Unfortunately, Apple (a major late-comer to RSS) doesn't seem to understand that "RSS content search, aggregation and transformation" is practically the same thing as "XML search, aggregation and transformation". Subject matter covered extensively by XML based languages such as XSLT, XPath, XPointer, and XQuery.

Without XML there would be no RSS (as we know it today), and without RSS there would be no Blogosphere.

Repurposing Blogosphere content isn't a novel invention at all. Therefore, filing a patent along such lines is simply uncool by Apple's standards (like the inextricable binding of iWeb to .mac that was touted as innovative and open).

Final note: this blog is driven by a database engine that has understood XML for a long time. This blog has been my live demo of this fact since its inception. Here are a few things that it has done for a very long time (talking prior art here):

- Repurpose content on the fly from SQL and XML data sources to produce all the syndication and subscription gems you see on the Blog Home Page

- Offer a search feature that enables visitors to query blog archives using Free Text, XQuery, XPath (all transformation technologies alongside XSLT).

- Produce Query Results that are "Open" for reuse outside the domain of this blog using standard syndication formats.

Ajax-S: Ajaxian slideshow software: "The idea came to me because I wanted a lightweight slideshow based on HTML, CSS and JavaScript, but I also wanted to separate the data of each page from the actual code that presents it. Therefore, I decided to move the data into an XML file and then use AJAX to retrieve it. The name AJAX-S is short for AJAX-Slides (or Asynchronous JavaScript and XML Slides, if you want to)."

(Via Ajaxian Blog.)

AJAX is clearly illuminating one of my pet issues: Separation of Application/Service Logic and Data. Even better, the concept of XML instance data is gradually getting much clearer. AJAX has created context for validating the concept of browser hosted Rich Internet Applications (RIA).

AJAX has become a widely accepted framework for the InternetOS that facilitates Rich Internet Application development using Web 2.0 (and beyond) APIs.

This Week’s Semantic Web:

"Ok, my first attempt at a round-up (in response to Phil’s observation of Planetary damage). Thanks to the conference there’s loads more here than there’s likely to be subsequent weeks, although it’s still only a fairly random sample and some of the links here are to heaps of other resources…
Incidentally, if anyone’s got a list/links for SemWeb-related blogs that aren’t on Planet RDF, I’d be grateful for a pointer. PS. Ok, I forget… are there any blogs that aren’t on Dave’s list yet..?

Quote of the week:

In the Semantic Web, it is not the Semantic which is new, it is the Web which is new.

- Chris Welty, IBM (lifted from TimBL’s slides)

Events

• 4th International Semantic Web Conference - happened this week, see : ISWC2005 Semantic Bank
• Semantic Desktop Workshop, 9-13 December 2005, Berlin
• Semantic Web Applications and Perspectives/Workshop (SWAP2005), 14-16 December, 2005
• Jena User Conference - May 10-11 2006, Bristol UK

Docs etc

• Conference highlights on the #swig chump: 2005-11-06, -07, -08, -09, -10; Ian’s notes; John’s resources; Leo’s stories; Uldis’ call to action; del.icio.us/iswc2005; flickr/iswc2005; foaf-moblog.
• Slides from Sir TimBL’s conference keynotes: Semantic Web for the Industry, Putting the Web back in Semantic Web
• Daniel Weitzner’s keynote: Privacy, Provenance, Property and Personhood
• Long-time SW researcher Stefan Decker now has a blog, inspirationally entitled Stefan Decker on the Semantic Web. (Stefan’s one of the head honchos at DERI). Sample snippet:
  

  I just noticed the article from Dan Zambonini ‘Is Web 2.0 killing the Semantic Web?‘. From my perspective the article shows a misconception that people seems to have around the Semantic Web: the Semantic Web effort itself is not provide applications (like the Web 2.0 meme indicates) - it rather provides standards to interlink applications.

• Leigh Dodds has two pieces demonstrating neat facilities offered by ARQ the SPARQL query API for Java: parameterised queries and extension functions.
• A new W3C Working Group has been chartered: Rule Interchange Format WG - ’ to produce a core rule language plus extensions which together allow rules to be translated between rule languages and thus transferred between rule systems.’. As noted by dajobe, phase 1 includes making a new XML syntax for RDF…
• UMBC Semantic Web Reference Card - if you only print one thing this year…or did you already do the SPARQL Reference card..?
• WebDescription - root wiki page for collecting notes on web description languages (ESW Wiki, announcement)
• Bot - IRC/Jabber chat bots that are either in use by Semantic Web developers or use Semantic Web technologies (ESW Wiki)
• microformat FAQs for RDF fans (ESW Wiki)
• W3C working draft : WSDL 2.0 - RDF Mapping
• SKOS (Simple Knowledge Organisation System) updated drafts: SKOS Core Vocabulary Specification, SKOS Core Guide
• working draft: SPARQL Protocol for RDF Using WSDL 1.1
• A relational algebra for SPARQL, Note on database layouts for SPARQL datastores (PDFs, Richard Cyganiak, HP)
• Amateur Fiction Online - The Web of Community Trust A Case Study in Community Focused Design for the SemanticWeb (PDF)
• Building a Semantic Wiki - IEEE article. See also: SemperWiki - Semantic Personal Wiki, WikSAR - Towards a Semantic Wiki Experience
  

Software and stuff

• Semantic Web Challenge applications (winner: CONFOTO - congrats bengee!)
• Piggy Bank 2.1.1 released.
• IRIS is a semantic desktop application framework that enables users to create a ‘personal map’ across their office-related information objects. IRIS includes a machine-learning platform to help automate this process. It provides ‘dashboard’ views, contextual navigation, and relationship-based structure across an extensible suite of office applications, including a calendar, web and file browser, e-mail client, and instant messaging client.
  (open source release due Jan 2006)
• MKSearch - ‘A new kind of search engine’ - RDF-backed (Sesame) with Web crawler, extracts and indexes metadata.
• FOAFRealm - Our goal is to design and implement D-FOAF, a distributed authentication and trust infrastructure without a centralised authority. D-FOAF will be a backbone for trust applications based on social relationships and will establish identity of users similar to the way we establish identify and trust in real life.
• Perl Net::Flickr::RDF
• WordPress SIOC (Semantically-Interlinked Online Communities) plugin updated (just copy wp-sioc.php into the root of your WP install and it just works)
• OntoMedia is intended for the representation of heterogenous media through description of the semantic content of that media. The representation may be limited to the description of some or all of the elements contained within the source or may include information regarding the narrative relationship that these elements have both to the media and to each other.
• mSpace is an interaction model to help explore relationships in information - ‘Imagine Google on iTunes’

Blog post title of the week:

Don’t give me that monkey-ass Web 1.0, either

- Uche Ogbuji

Also…a new threat to Semantic Web developers has been discovered: typhoid!, and the key to the Web’s full potential is…Tetris."

iPodder Lemon changes its name to...: "iPodder Lemon changes its name to Juice Receiver. 'Apple went after us...'"

(Via Scripting News.)

iPodder Lemon changes its name to...: "iPodder Lemon changes its name to Juice Receiver. 'Apple went after us...'"

(Poster: Dave Winer Via Scripting News.)

Solutions to allow XMLHttpRequest to talk to external services: "

Over on XML.com they published Fixing AJAX: XmlHttpRequest Considered Harmful.

This article discusses a few ways to get around the security constraints that we have to live with in the browsers theses days, in particular, only being able to talk to your domain via XHR.

The article walks you through three potential solutions:

1. Application proxies. Write an application in your favorite programming language that sits on your server, responds to XMLHttpRequests from users, makes the web service call, and sends the data back to users.
2. Apache proxy. Adjust your Apache web server configuration so that XMLHttpRequests can be invisibly re-routed from your server to the target web service domain.
3. Script tag hack with application proxy (doesn't use XMLHttpRequest at all). Use the HTML script tag to make a request to an application proxy (see #1 above) that returns your data wrapped in JavaScript. This approach is also known as On-Demand JavaScript.

I can't wait for Trusted Relationships within the browser - server infrastructure.

With respect to Apache proxies, these things are priceless. I recently talked about them in relation to Migrating data centers with zero downtime.

What do you guys think about this general issue? Have you come up with any interesting solutions? Any ideas on how we can keep security, yet give us the freedom that we want?

(Via Ajaxian Blog.)

Well here is what I think (actually know):

Our Virtuoso Universal Server has been sitting waiting to deliver this for years (for the record see the Virtuoso 2000 Press Release). Virtuoso can proxy for disparate data sources and expose disparate data as Well-Formed XML using an array of vocabularies (you experience this SQL-XML integration on the fly every time you interact with various elements of my public blog).

Virtuoso has always been able to expose Application Logic as SOAP and/or RESTful/RESTian style XML Web Services. This blog's search page is a simple demo of this capability.

Virtuoso is basically a Junction Box / Aggregator / Proxy for disparate Data, Applications, Services, and BPEL compliant business processes. AJAX clients talk to this single multi-purpose server which basically acts as a conduit to content/data, services, and processes (which are composite services).

BTW - there is a lot more, but for now, thou shall have to seek in order to find :-)

Quick Example using my blog:

1. My Blog's Search Page (note it support Full Text and XPath/XQuery)

2. Search on pattern 'Web 2.0' via my Blog's Search Engine

3. Hactivism" regarding this matter. Certainly worth a full-post-scrape for my ongoing content annotation efforts (see Linkblog and BlogSummary).

Digest the rest of Dare's post:

Clone the Google APIs: Kill That Noise: "

Yesterday Dave Winer wrote in a post about cloning the Google API Dave Winer wrote

Let's make the Google API an open standard. Back in 2002, Google took a bold first step to enable open architecture search engines, by creating an API that allowed developers to build applications on top of their search engine. However, there were severe limits on the capacity of these applications. So we got a good demo of what might be, now three years later, it's time for the real thing.



and earlier that


If you didn't get a chance to hear yesterday's podcast, it recommends that Microsoft clone the Google API for search, without the keys, and without the limits. When a developer's application generates a lot of traffic, buy him a plane ticket and dinner, and ask how you both can make some money off their excellent booming application of search. This is something Google can't do, because search is their cash cow. That's why Microsoft should do it. And so should Yahoo. Also, there's no doubt Google will be competing with Apple soon, so they should be also thinking about ways to devalue Google's advantage.

This doesn't seem like a great idea to me for a wide variety of reasons but first, let's start with a history lesson before I tackle this specific issue

A Trip Down Memory Lane
This history lesson used to be in is in a post entitled The Tragedy of the API by Evan Williams but seems to be gone now. Anyway, back in the early days of blogging the folks at Pyra [which eventually got bought by Google] created the Blogger API for their service. Since Blogspot/Blogger was a popular service, a the number of applications that used the API quickly grew. At this point Dave Winer decided that since the Blogger API was so popular he should implement it in his weblogging tools but then he decided that he didn't like some aspects of it such as application keys (sound familiar?) and did without them in his version of the API. Dave Winer's version of the Blogger API became the MetaWeblog API. These APIs became de facto standards and a number of other weblogging applications implemented them.

After a while, the folks at Pyra decided that their API needed to evolve due to various flaws in its design. As Diego Doval put it in his post a review of blogging APIs, The Blogger API is a joke, and a bad one at that. This lead to the creation of the Blogger API 2.0. At this point a heated debate erupted online where Dave Winer berated the Blogger folks for deviating from an industry standard. The irony of flaming a company for coming up with a v2 of their own API seemed to be lost on many of the people who participated in the debate. Eventually the Blogger API 2.0 went nowhere.

Today the blogging API world is a few de facto standards based on a hacky API created by a startup a few years ago, a number of site specific APIs (LiveJournal API, MovableType API, etc) and a number of inconsistently implemented versions of the Atom API.


On Cloning the Google Search API
To me the most salient point in the hijacking of the Blogger API from Pyra is that it didn't change the popularity of their service or even make Radio Userland (Dave Winer's product) catch up to them in popularity. This is important to note since this is Dave Winer's key argument for Microsoft cloning the Google API.

Off the top of my head, here are my top three technical reasons for Microsoft to ignore the calls to clone the Google Search APIs


1. Difference in Feature Set: The features exposed by the API do not run the entire gamut of features that other search engines may want to expose. Thus even if you implement something that looks a lot like the Google API, you'd have to extend it to add the functionality that it doesn't provide. For example, compare the features provided by the Google API to the features provided by the Yahoo! search API. I can count about half a dozen features in the Yahoo! API that aren't in the Google API.

2. Difference in Technology Choice: The Google API uses SOAP. This to me is a phenomenally bad technical decision because it raises the bar to performing a basic operation (data retrieval) by using a complex technology. I much prefer Yahoo!'s approach of providing a RESTful API and MSN Windows Live Search's approach of providing RSS search feeds and a SOAP API for the folks who need such overkill.
   

3. Unreasonable Demands: A number of Dave Winer's demands seem contradictory. He asks companies to not require application keys but then advises them to contact application developers who've built high traffic applications about revenue sharing. Exactly how are these applications to be identified without some sort of application ID? As for removing the limits on the services? I guess Dave is ignoring the fact that providing services costs money, which I seem to remember is why he sold weblogs.com to Verisign for a few million dollars. I do agree that some of the limits on existing search APIs aren't terribly useful. The Google API limit of 1000 queries a day seems to guarantee that you won't be able to power a popular application with the service.
   

4. Lack of Innovation: Copying Google sucks.
   

(Via Dare Obasanjo aka Carnage4Life.)

Two mapping/geocoding APIs have been recently added to the database:

• Ontok: Provides a mechanism for geocoding the latitude and longitude of any US address. The data is based on the US Census 2004 Second Edition TIGER/Line® Files. Here’s an example of the White House. The service is accessible via REST, SOAP and JavaScript APIs. See also the nice mashup UI with Google Maps where you can click on the lower left Input Addresses to navigate.
• ZeeMaps: If you need to geocode addresses outside of the US, ZeeMaps can help. Given an international city, state (optional), and country combination, the service will return a set of latitude, longitudes (along with corrected city, state, country information). They offer REST, XML-RPC and SOAP interfaces.

(Via ProgrammableWeb.com.)

Anyway, Marc's article is a very refreshing read because it provides a really good insight into the general landscape of a rapidly evolving Web alongside genuine appreciation of our broader timeless pursuit of "Openness".

To really help this document provide additional value have scrapped the content of the original post and dumped it below so that we can appreciate the value of the links embedded within the article (note: thanks to Virtuoso I only had to paste the content into my blog, the extraction to my Linkblog and Blog Summary Pages are simply features of my Virtuoso based Blog Engine):

Breaking the Web Wide Open! (complete story)

Even the web giants like AOL, Google, MSN, and Yahoo need to observe these open standards, or they'll risk becoming the "walled gardens" of the new web and be coolio no more.

Marc Canter [Broadband Mechanics, Inc.] | POSTED: 09.26.05 @12:00

Editorial Note: Several months ago, AlwaysOn got a personal invitation from Yahoo founder Jerry Yang "to see and give us feedback on our new social media product, y!360." We were happy to oblige and dutifully showed up, joining a conference room full of hard-core bloggers and new, new media types. The geeks gave Yahoo 360 an overwhelming thumbs down, with comments like, "So the only services I can use within this new network are Yahoo services? What if I don't use Yahoo IM?" In essence, the Yahoo team was booed for being "closed web," and we heartily agreed. With Yahoo 360, Yahoo continues building its own "walled garden" to control its 135 million customers—an accusation also hurled at AOL in the early 1990s, before AOL migrated its private network service onto the web. As the  Economist recently noted, "Yahoo, in short, has old media plans for the new-media era." The irony to our view here is, of course, that today's AO Network is also a "closed web." In the end, Mr. Yang's thoughtful invitation and our ensuing disappointment in his new service led to the assignment of this article. It also confirmed our existing plan to completely revamp the AO Network around open standards. To tie it all together, we recruited the chief architect of our new site, the notorious Marc Canter, to pen this piece. We look forward to our reader feedback. Breaking the Web Wide Open! By Marc Canter For decades, "walled gardens" of proprietary standards and content have been the strategy of dominant players in mainframe computer software, wireless telecommunications services, and the World Wide Web—it was their successful lock-in strategy of keeping their customers theirs. But like it or not, those walls are tumbling down. Open web standards are being adopted so widely, with such value and impact, that the web giants—Amazon, AOL, eBay, Google, Microsoft, and Yahoo—are facing the difficult decision of opening up to what they don't control. The online world is evolving into a new open web (sometimes called the Web 2.0), which is all about being personalized and customized for each user. Not only open source software, but open standards are becoming an essential component. Many of the web giants have been using open source software for years. Most of them use at least parts of the LAMP (Linux, Apache, MySQL, Perl/Python/PHP) stack, even if they aren't well-known for giving back to the open source community. For these incumbents that grew big on proprietary web services, the methods, practices, and applications of open source software development are difficult to fully adopt. And the next open source movements—which will be as much about open standards as about code—will be a lot harder for the incumbents to exploit. While the incumbents use cheap open source software to run their back-ends systems, their business models largely depend on proprietary software and algorithms. But our view a new slew of open software, open protocols, and open standards will confront the incumbents with the classic Innovator's Dilemma.  Should they adopt these tools and standards, painfully cannibalizing their existing revenue for a new unproven concept, or should they stick with their currently lucrative model with the risk that eventually a bunch of upstarts eat their lunch? Credit should go to several of the web giants who have been making efforts to "open up." Google, Yahoo, eBay, and Amazon all have Open APIs (Application Programming Interfaces) built into their data and systems. Any software developer can access and use them for whatever creative purposes they wish. This means that the API provider becomes an open platform for everyone to use and build on top of. This notion has expanded like wildfire throughout the blogosphere, so nowadays, Open APIs are pretty much required. Other incumbents also have open strategies. AOL has got the RSS religion, providing a feedreader and RSS search in order to escape the "walled garden of content" stigma. Apple now incorporates podcasts, the "personal radio shows" that are latest rage in audio narrowcasting, into iTunes. Even Microsoft is supporting open standards, for example by endorsing SIP (Session Initiation Protocol) for internet telephony and conferencing over Skype's proprietary format or one of its own devising. But new open standards and protocols are in use, under construction, or being proposed every day, pushing the envelope of where we are right now. Many of these standards are coming from startup companies and small groups of developers, not from the giants. Together with the Open APIs, those new standards will contribute to a new, open infrastructure. Tens of thousands of developers will use and improve this open infrastructure to create new kinds of web-based applications and services, to offer web users a highly personalized online experience. A Brief History of Openness At this point, I have to admit that I am not just a passive observer, full-time journalist or "just some blogger"—but an active evangelist and developer of these standards. It's the vision of "open infrastructure" that's driving my company and the reason why I'm writing this article. This article will give you some of the background behind on these standards, and what the evolution of the next generation of open standards will look like. Starting back in the 1980s, establishing a software standard was a key strategy for any software company. My former company, MacroMind (which became Macromedia), achieved this goal early on with Director. As Director evolved into Flash, the world saw that other companies besides Microsoft, Adobe, and Apple could establish true cross-platform, independent media standards. Then Tim Berners-Lee and Marc Andreessen came along, and changed the rules of the software business and of entrepreneurialism. No matter how entrenched and "standardized" software was, the rug could still get pulled out from under it. Netscape did it to Microsoft, and then Microsoft did it back  to Netscape. The web evolved, and lots of standards evolved with it. The leading open source standards (such as the LAMP stack) became widely used alternatives to proprietary closed-source offerings. Open standards are more than just technology. Open standards mean sharing, empowering, and community support. Someone floats a new idea (or meme) and the community runs with it – with each person making their own contributions to the standard – evolving it without a moment's hesitation about "giving away their intellectual property." One good example of this was Dave Sifry, who built the Technorati blog-tracking technology inspired by the Blogging Ecosystem, a weekend project by young hacker Phil Pearson. Dave liked what he saw and he ran with it—turning Technorati into what it is today. Dave Winer has contributed enormously to this area of open standards. He defined and personally created several open standards and protocols—such as RSS, OPML, and XML-RPC. Dave has also helped build the blogosphere through his enthusiasm and passion. By 2003, hundreds of programmers were working on creating and establishing new standards for almost everything. The best of these new standards have evolved into compelling web services platforms – such as del.icio.us, Webjay, or Flickr. Some have even spun off formal standards – like XSPF (a standard for playlists) or instant messaging standard XMPP (also known as Jabber). Today's Open APIs are complemented by standardized Schemas—the structure of the data itself and its associated meta-data. Take for example a podcasting feed. It consists of: a) the radio show itself, b) information on who is on the show, what the show is about and how long the show is (the meta-data) and also c) API calls to retrieve a show (a single feed item) and play it from a specified server. The combination of Open APIs, standardized schemas for handling meta-data, and an industry which agrees on these standards are breaking the web wide open right now. So what new open standards should the web incumbents—and you—be watching? Keep an eye on the following developments: Identity Attention Open Media Microcontent Publishing Open Social Networks Tags Pinging Routing Open Communications Device Management and Control 1. Identity Right now, you don't really control your own online identity. At the core of just about every online piece of software is a membership system. Some systems allow you to browse a site anonymously—but unless you register with the site you can't do things like search for an article, post a comment, buy something, or review it. The problem is that each and every site has its own membership system. So you constantly have to register with new systems, which cannot share data—even you'd want them to. By establishing a "single sign-on" standard, disparate sites can allow users to freely move from site to site, and let them control the movement of their personal profile data, as well as any other data they've created. With Passport, Microsoft unsuccessfully attempted to force its proprietary standard on the industry. Instead, a world is evolving where most people assume that users want to control their own data, whether that data is their profile, their blog posts and photos, or some collection of their past interactions, purchases, and recommendations. As long as users can control their digital identity, any kind of service or interaction can be layered on top of it. Identity 2.0 is all about users controlling their own profile data and becoming their own agents. This way the users themselves, rather than other intermediaries, will profit from their ID info. Once developers start offering single sign-on to their users, and users have trusted places to store their data—which respect the limits and provide access controls over that data, users will be able to access personalized services which will understand and use their personal data. Identity 2.0 may seem like some geeky, visionary future standard that isn't defined yet, but by putting each user's digital identity at the core of all their online experiences, Identity 2.0 is becoming the cornerstone of the new open web. The Initiatives: Right now, Identity 2.0 is under construction through various efforts from Microsoft (the "InfoCard" component built into the Vista operating system and its "Identity Metasystem"), Sxip Identity, Identity Commons, Liberty Alliance, LID (NetMesh's Lightweight ID), and SixApart's OpenID. More Movers and Shakers: Identity Commons and Kaliya Hamlin, Sxip Identity and Dick Hardt, the Identity Gang and Doc Searls, Microsoft's Kim Cameron, Craig Burton, Phil Windley, and Brad Fitzpatrick, to name a few. 2. Attention How many readers know what their online attention is worth? If you don't, Google and Yahoo do—they make their living off our attention. They know what we're searching for, happily turn it into a keyword, and sell that keyword to advertisers. They make money off our attention. We don't. Technorati and friends proposed an attention standard, Attention.xml, designed to "help you keep track of what you've read, what you're spending time on, and what you should be paying attention to." AttentionTrust is an effort by Steve Gillmor and Seth Goldstein to standardize on how captured end-user performance, browsing, and interest data are used. Blogger Peter Caputa gives a good summary of AttentionTrust: "As we use the web, we reveal lots of information about ourselves by what we pay attention to. Imagine if all of that information could be stored in a nice neat little xml file. And when we travel around the web, we can optionally share it with websites or other people. We can make them pay for it, lease it ... we get to decide who has access to it, how long they have access to it, and what we want in return. And they have to tell us what they are going to do with our Attention data." So when you give your attention to sites that adhere to the AttentionTrust, your attention rights (you own your attention, you can move your attention, you can pay attention and be paid for it,  and you can see how your attention is used) are guaranteed. Attention data is crucial to the future of the open web, and Steve and Seth are making sure that no one entity or oligopoly controls it. Movers and Shakers: Steve Gillmor, Seth Goldstein, Dave Sifry and the other Attention.xml folks. 3. Open Media Proprietary media standards—Flash, Windows Media, and QuickTime, to name a few —helped liven up the web. But they are proprietary standards that try to keep us locked in, and they weren't created from scratch to handle today's online content. That's why, for many of us, an Open Media standard has been a holy grail. Yahoo's new Media RSS standard brings us one step closer to achieving open media, as do Ogg Vorbis audio codecs, XSPF playlists, or MusicBrainz. And several sites offer digital creators not only a place to store their content, but also to sell it. Media RSS (being developed by Yahoo with help from the community) extends RSS and combines it with "RSS enclosures" —adds metadata to any media item—to create a comprehensive solution for media "narrowcasters." To gain acceptance for Media RSS, Yahoo knows it has to work with the community. As an active member of this community, I can tell you that we'll create Media RSS equivalents for rdf (an alternative subscription format) and Atom (yet another  subscription format), so no one will be able to complain that Yahoo is picking sides in format wars. When Yahoo announced the purchase of Flickr, Yahoo founder Jerry Yang insinuated that Yahoo is acquiring "open DNA" to turn Yahoo into an open standards player. Yahoo is showing what happens when you take a multi-billion dollar company and make openness one of its core values—so Google, beware, even if Google does have more research fellows and Ph.D.s. The open media landscape is far and wide, reaching from game machine hacks and mobile phone downloads to PC-driven bookmarklets, players, and editors, and it includes many other standardization efforts. XSPF is an open standard for playlists, and MusicBrainz is an alternative to the proprietary (and originally effectively stolen) database that Gracenote licenses. Ourmedia.org is a community front-end to Brewster Kahle's Internet Archive. Brewster has promised free bandwidth and free storage forever to any content creators who choose to share their content via the Internet Archive. Ourmedia.org is providing an easy-to-use interface and community to get content in and out of the Internet Archive, giving ourmedia.org users the ability to share their media anywhere they wish, without being locked into a particular service or tool. Ourmedia plans to offer open APIs and an open media registry that interconnects other open media repositories into a DNS-like registry (just like the www domain system), so folks can browse and discover open content across many open media services. Systems like Brightcove and Odeo support the concept of an open registry, and hope to work with digital creators to sell their work to fulfill the financial aspect of the "Long Tail." More Movers and Shakers: Creative Commons, the Open Media Network, Jay Dedman, Ryanne Hodson, Michael Verdi, Eli Chapman, Kenyatta Cheese, Doug Kaye, Brad Horowitz, Lucas Gonze, Robert Kaye, Christopher Allen, Brewster Kahle, JD Lasica, and indeed, Marc Canter, among others. 4. Microcontent Publishing Unstructured content is cheap to create, but hard to search through. Structured content is expensive to create, but easy to search. Microformats resolve the dilemma with simple structures that are cheap to use and easy to search. The first kind of widely adopted microcontent is blogging. Every post is an encapsulated idea, addressable via a URL called a permalink. You can syndicate or subscribe to this microcontent using RSS or an RSS equivalent, and news or blog aggregators can then display these feeds in a convenient readable fashion. But a blog post is just a block of unstructured text—not a bad thing, but just a first step for microcontent. When it comes tostructured data, such as personal identity profiles, product reviews, or calendar-type event data, RSS was not designed to maintain the integrity of the structures. Right now, blogging doesn't have the underlying structure necessary for full-fledged microcontent publishing. But that will change. Think of local information services (such as movie listings, event guides, or restaurant reviews) that any college kid can access and use in her weekend programming project to create new services and tools. Today's blogging tools will evolve into microcontent publishing systems, and will help spread the notion of structured data across the blogosphere. New ways to store, represent and produce microcontent will create new standards, such as Structured Blogging and Microformats. Microformats differ from RSS feeds in that you can't subscribe to them. Instead, Microformats are embedded into webpages and discovered by search engines like Google or Technorati. Microformats are creating common definitions for "What is a review or event? What are the specific fields in the data structure?" They can also specify what we can do with all this information.OPML (Outline Processor Markup Language) is a hierarchical file format for storing microcontent and structured data. It was developed by Dave Winer of RSS and podcast fame. Events are one popular type of microcontent. OpenEvents is already working to create shared databases of standardized events, which would get used by a new generation of event portals—such as Eventful/EVDB, Upcoming.org, and WhizSpark. The idea of OpenEvents is that event-oriented systems and services can work together to establish shared events databases (and associated APIs) that any developer could then use to create and offer their own new service or application. OpenReviews is still in the conceptual stage, but it would make it possible to provide open alternatives to closed systems like Epinions, and establish a shared database of local and global reviews. Its shared open servers would be filled with all sorts of reviews for anyone to access. Why is this important? Because I predict that in the future, 10 times more people will be writing reviews than maintaining their own blog. The list of possible microcontent standards goes on: OpenJobpostings, OpenRecipes, and even OpenLists. Microsoft recently revealed that it has been working on an important new kind of microcontent: Lists—so OpenLists will attempt to establish standards for the kind of lists we all use, such as lists of Links, lists of To Do Items, lists of People, Wish Lists, etc. Movers and Shakers: Tantek Çelik and Kevin Marks of Technorati, Danny Ayers, Eric Meyer, Matt Mullenweg, Rohit Khare, Adam Rifkin, Arnaud Leene, Seb Paquet, Alf Eaton, Phil Pearson, Joe Reger, Bob Wyman among others. 5. Open Social Networks I'll never forget the first time I met Jonathan Abrams, the founder of Friendster. He was arrogant and brash and he claimed he "owned"  all his users, and that he was going to monetize them and make a fortune off them. This attitude robbed Friendster of its momentum, letting MySpace, Facebook, and other social networks take Friendster's place. Jonathan's notion of social networks as a way to control users is typical of the Web 1.0 business model and its attitude towards users in general. Social networks have become one of the battlegrounds between old and new ways of thinking. Open standards for Social Networking will define those sides very clearly. Since meeting Jonathan, I have been working towards finding and establishing open standards for social networks. Instead of closed, centralized social networks with 10 million people in them, the goal is making it possible to have 10 million social networks that each have 10 people in them. FOAF (which stands for Friend Of A Friend, and describes people and relationships in a way that computers can parse) is a schema to represent not only your personal profile's meta-data, but your social network as well. Thousands of researchers use the FOAF schema in their "Semantic Web" projects to connect people in all sorts of new ways. XFN is a microformat standard for representing your social network, while vCard (long familiar to users of contact manager programs like Outlook) is a microformat that contains your profile information. Microformats are baked into any xHTML webpage, which means thatany blog, social network page, or any webpage in general can "contain" your social network in it—and be used byany compatible tool, service or application. PeopleAggregator is an earlier project now being integrated into open content management framework Drupal. The PeopleAggregator APIs will make it possible to establish relationships, send messages, create or join groups, and post between different social networks. (Sneak preview: this technology will be available in the upcoming GoingOn Network.) All of these open social networking standards mean that inter-connected social networks will form a mesh that will parallel the blogosphere. This vibrant, distributed, decentralized world will be driven by open standards: personalized online experiences are what the new open web will be all about—and what could be more personalized than people's networks? Movers and Shakers: Eric Sigler, Joel De Gan, Chris Schmidt, Julian Bond, Paul Martino, Mary Hodder, Drummond Reed, Dan Brickley, Randy Farmer, and Kaliya Hamlin, to name a few. 6. Tags Nowadays, no self-respecting tool or service can ship without tags. Tags are keywords or phrases attached to photos, blog posts, URLs, or even video clips. These user- and creator-generated tags are an open alternative to what used to be the domain of librarians and information scientists: categorizing information and content using taxonomies. Tags are instead creating "folksonomies." The recently proposed OpenTags concept would be an open, community-owned version of the popular Technorati Tags service. It would aggregate the usage of tags across a wide range of services, sites, and content tools. In addition to Technorati's current tag features, OpenTags would let groups of people share their tags in "TagClouds." Open tagging is likely to include some of the open identity features discussed above, to create a tag system that is resilient to spam, and yet trustable across sites all over the web. OpenTags owes a debt to earlier versions of shared tagging systems, which include Topic Exchange and something called the k-collector—a knowledge management tag aggregator—from Italian company eVectors. Movers & Shakers: Phil Pearson, Matt Mower , Paolo Valdemarin, and Mary Hodder and Drummond Reed again, among others. 7. Pinging Websites used to be mostly static. Search engines that crawled (or "spidered") them every so often did a good enough job to show reasonably current versions of your cousin's homepage or even Time magazine's weekly headlines. But when blogging took off, it became hard for search engines to keep up. (Google has only just managed to offer blog-search functionality, despite buying Blogger back in early 2003.) To know what was new in the blogosphere, users couldn't depend on services that spidered webpages once in a while. The solution: a way for blogs themselves to automatically notify blog-tracking sites that they'd been updated. Weblogs.com was the first blog "ping service": it displayed the name of a blog whenever that blog was updated. Pinging sites helped the blogosphere grow, and more tools, services, and portals started using pinging in new and different ways. Dozens of pinging services and sites—most of which can't talk to each other—sprang up. Matt Mullenweg (the creator of open source blogging software WordPress) decided that a one-stop service for pinging was needed. He created Ping-o-Matic—which aggregates ping services and simplifies the pinging process for bloggers and tool developers. With Ping-o-Matic, any developer can alert all of the industry's blogging tools and tracking sites at once. This new kind of open standard, with shared infrastructure, is a critical to the scalability of Web 2.0 services. As Matt said: There are a number of services designed specifically for tracking and connecting blogs. However it would be expensive for all the services to crawl all the blogs in the world all the time. By sending a small ping to each service you let them know you've updated so they can come check you out. They get the freshest data possible, you don't get a thousand robots spidering your site all the time. Everybody wins. Movers and Shakers: Matt Mullenweg, Jim Winstead, Dave Winer 8. Routing Bloggers used to have to manually enter the links and content snippets of blog posts or news items they wanted to blog. Today, some RSS aggregators can send a specified post directly into an associated blogging tool: as bloggers browse through the feeds they subscribe to, they can easily specify and send any post they wish to "reblog" from their news aggregator or feed reader into their blogging tool. (This is usually referred to as "BlogThis.") As structured blogging comes into its own (see the section on Microcontent Publishing), it will be increasingly important to maintain the structural integrity of these pieces of microcontent when reblogging them. Promising standard RedirectThis will combine a "BlogThis"-like capability while maintaining the integrity of the microcontent. RedirectThis will let bloggers and content developers attach a simple "PostThis" button to their posts. Clicking on that button will send that post to the reader/blogger's favorite blogging tool. This favorite tool is specified at the RedirectThis web service, where users register their blogging tool of choice. RedirectThis also helps maintain the integrity and structure of microcontent—then it's just up to the user to prefer a blogging tool that also attains that lofty goal of microcontent integrity. OutputThis is another nascent web services standard, to let bloggers specify what "destinations" they'd like to have as options in their blogging tool. As new destinations are added to the service, more checkboxes would get added to their blogging tool—allowing them to route their published microcontent to additional destinations. Movers and Shakers: Michael Migurski, Lucas Gonze 9. Open Communications Likely, you've experienced the joys of finding friends on AIM or Yahoo Messenger, or the convenience of Skyping with someone overseas. Not that you're about to throw away your mobile phone or BlackBerry, but for many, also having access to Instant Messaging (IM) and Voice over IP (VoIP) is crucial. IM and VoIP are mainstream technologies that already enjoy the benefits of open standards. Entire industries are born—right this second—based around these open standards. Jabber has been an open IM technology for years—in fact, as XMPP, it was officially dubbed a standard by the IETF. Although becoming an official IETF standard is usually the kiss of death, Jabber looks like it'll be around for a while, as entire generations of collaborative, work-group applications and services have been built on top of its messaging protocol. For VoIP, Skype is clearly the leading standard today—though one could argue just how "open" it is (and defenders of the IETF's SIP standard often do). But it is free and user-friendly, so there won't be much argument from users  about it being insufficiently open. Yet there may be a cloud on Skype's horizon: web behemoth Google recently released a beta of Google Talk, an IM client committed to open standards. It currently supports XMPP, and will support SIP for VoIP calls. Movers and Shakers: Jeremie Miller, Henning Schulzrinne, Jon Peterson, Jeff Pulver 10. Device Management and Control To access online content, we're using more and more devices. BlackBerrys, iPods, Treos, you name it. As the web evolves, more and more different devices will have to communicate with each other to give us the content we want when and where we want it. No-one wants to be dependent on one vendor anymore—like, say, Sony—for their laptop, phone, MP3 player, PDA, and digital camera, so that it all works together. We need fully interoperable devices, and the standards to make that work. And to fully make use of how content is moving online content and innovative web services, those standards need to be open. MIDI (musical instrument digital interface), one of the very first open standards in music, connected disparate vendors' instruments, post-production equipment, and recording devices. But MIDI is limited, and MIDI II has been very slow to arrive. Now a new standard for controlling musical devices has emerged: OSC (Open SoundControl). This protocol is optimized for modern networking technology and inter-connects music, video and controller devices with "other multimedia devices." OSC is used by a wide range of developers, and is being taken up in the mainstream MIDI marketplace. Another open-standards-based device management technology is ZigBee, for building wireless intelligence and network monitoring into all kinds of devices. ZigBee is supported by many networking, consumer electronics, and mobile device companies.       · · · · · ·     The Change to Openness The rise of open source software and its "architecture of participation" are completely shaking up the old proprietary-web-services-and-standards approach. Sun Microsystems—whose proprietary Java standard helped define the Web 1.0—is opening its Solaris OS and has even announced the apparent paradox of an open-source Digital Rights Management system. Today's incumbents will have to adapt to the new openness of the Web 2.0. If they stick to their proprietary standards, code, and content, they'll become the new walled gardens—places users visit briefly to retrieve data and content from enclosed data silos, but not where users "live." The incumbents' revenue models will have to change. Instead of "owning" their users, users will know they own themselves, and will expect a return on their valuable identity and attention. Instead of being locked into incompatible media formats, users will expect easy access to digital content across many platforms. Yesterday's web giants and tomorrow's users will need to find a mutually beneficial new balance—between open and proprietary, developer and user, hierarchical and horizontal, owned and shared, and compatible and closed. Marc Canter is an active evangelist and developer of open standards. Early in his career, Marc founded MacroMind, which became Macromedia. These days, he is CEO of Broadband Mechanics, a founding member of the Identity Gang and of ourmedia.org. Broadband Mechanics is currently developing the GoingOn Network (with the AlwaysOn Network), as well as an open platform for social networking called the PeopleAggregator. A version of the above post appears in the Fall 2005 issue of AlwaysOn's quarterly print blogozine, and ran as a four-part series on the AlwaysOn Network website.

(Via Marc's Voice.)

[You don’t expect me to work out the CSS right after making it semantic, do you?]

Shift to another universe. It’s sometime in the late 1990’s. Ramanathan Guha, Tim Bray, Dave Winer, Tantek Çelik, Dan Libby and Dan Connolly are sharing a jacuzzi*. As they sip Marghueritas, their conversation goes like this:

• DanL
  

  So, we’ve got this idea for publishing content that’s a bit like CDF, but we’ve made the system more of a service than just a desktop thing.

• Guha
  

  Sounds cool. Might be a good fit with this RDF thing I’ve been working on.

• Dave
  

  Hmm, Dan’s stuff does sound cool, but with all due respect dude, RDF does seem a bit complicated. I really don’t think the folks out in userland would get it. And they majored in graphs.

• Tim
  

  Maybe we could make it a bit more straightforward, you know, like put pointy brackets around it?

• Dave
  

  Straightforward’s good. Better still, simple. They like simple.

• Tantek
  

  But what about the rest of the Web, you know, like HTML?

• DanL
  

  Hmm, but how do we do the timestamping kind of thing, and wrap it up in a ‘microposty’ way, the things that makes this distribution mode work?

• Guha
  

  Yeah, metadata is cool. Keep the metadata.

• Tim
  

  Not cheap though. The Web must be cheap. Did Andreesen show you his pictures..?

• Dave
  

  …’Microposty’? you mean like my newsletter thing, but on the Web?

• DanL
  

  Yep, like Cool Diary Entry of the Day

• Tim
  

  But do we really need 1000 pages of spec for that?

• Tantek
  

  …Incidentally, did you see my Box Model Hack?

• Guha
  

  Yup.

• DanL
  

  Yup.

• Tim
  

  Yup.

• Dave
  

  Yup. I explained that on DaveNet last year.

• MarcC
  

  Hey! I’ve got it: ‘MyDigitalCocktail’..?

• DanC
  

  Hang on, that gives me an idea…

There was a tangible outcome to this conversation: a document format which supports content and unambiguous, explicit, data and metadata, timestamping and much, much more. It’s viewable in a regular browser. Can be syndicated; can be aggregated. Unlike forgetful RSS, archives are almost always retrievable using regular HTTP methods. In this universe there was no RSS. No syndication wars. No talking-at-cross-purposes conflict between docheads and dataheads, syntax fans and model fans. No-one had to publish simple data in Byzantine RDF/XML. No-one had to deal with doubly-escaped content and silent data loss. There was no need for any new format for business cards, calendars, blogs, link lists, reviews, pet profiles. XHTML with CSS was more than enough. DanL got the MyNetscape he wanted. Tim got the simple, tight format he wanted. Guha got the AI. Tantek got to do presentations in a cool black raincoat. DanC finally got his schedule on his Palm Pilot. Dave got the credit. MarcC got the parasols and a grass skirt none of the others would admit to having brought.

Shift back to this universe. Check out hAtom. It’s not finished yet, but David’s been methodically working through the (utterly sound) microformats process. Looks good to me.

* apologies for the imagery, but how else do think Silicon Valley might seem to someone raised in the cowpat-coated hills of Derbyshire?

PS. Apologies to everyone mentioned. And before you suggest it, blogging *is* therapy.

(Via Raw.)

The future of the Web is Semantic: "

A nice quick overview (if you don’t mind the RDF/XML approach) at IBM developerWorks: The future of the Web is Semantic

(Via Raw.)

• Services, not packaged software, with cost-effective scalability
• Control over unique, hard-to-recreate data sources that get richer as more people use them
• Trusting users as co-developers
• Harnessing collective intelligence
• Leveraging the long tail through customer self-service
• Software above the level of a single device
• Lightweight user interfaces, development models, AND business models

• Exposes Web services that can be accessed on any device or platform by any developer or user. RSS feeds, RESTful APIs and SOAP APIs are all examples of Web services.
• Harnesses the collective knowledge of its user base to benefit users
• Leverages the long tail through customer self-service

Web 2.0 is a web of executable service invocation endpoints (those Web Services URIs) and well-formed content (all of that RSS, Atom, RDF, XHTML, etc. based Web Content out on the NET). The executable service invocation endpoints and well-formed content are accessible via URIs.

Put in even simpler terms, Web 2.0 is an incarnation of the web defined by URIs for invoking Web Services and/or consuming or syndicating well-formed content.

Looks like I've self edited my own definition in the process. :-)

If you don't grok this definition then consider using it as a trigger for taking a closer look at the dynamics that genuinely differentiate Web 1.0 and Web 2.0.

Web 1.0 was about web sites geared towards an interaction with human beings as opposed to computers. In a sense this mirrors the difference between HTML and XML.

A simple example (purchasing a book):

amazon.com provides value to you by enabling you to search and purchase the desired book online via the site http://www.amazon.com.

In the Web 1.0 era the process of searching for your desired book, and then eventually purchasing the book in question, required visible interaction with the site http://www.amazon.com. In today's Web 2.0 based Web the process of discovering a catalog of books, searching for your particular book of interest, and eventually purchasing the book, occurs via Web Services which amazon has chosen to expose via an executable endpoint (the Web point of presence for exposing its Web Services).

Direct interaction via http://www.amazon.com is no longer required. A weblog can quite easily associate keywords, tags, and post categories with items in amazon.com's catalogs. In addition, weblogs can also act as entry points for consuming the amazon.com value proposition (making books available for purchase online), by enabling you to purchase a book directly from the weblog (assuming the blog owner is an amazon associate etc..). Now compare the impact of this kind of value discovery and consumption cycle driven by software to the same process driven by humans interaction with a static or dynamic HTML page (Web 1.0 site).

To surmise, Web 2.0 is a reflection of the potential of XML expressed through the collective impact of Web Services (XML based distributed computing) and Well-formed Content (Blogosphere, Wikisphere, XHTML micro content etc.). The potential simply comes down to the ability to ultimately connect events, triggers, impulses (chatter, conversation, etc.), and data in general via URIs.

Let's never forget that XML is the reason why we have a blogosphere (RSS/Atom/RDF are applications of XML). Likewise, XML is also the reason why we have Web Services (doesn't matter what format).

As I have stated in the past, we must go by Web 2.0 en route what is popularly referred to as the Semantic Web (it will be known by another name by the time we get there; 3.0 or 4.0, who knows or cares?). At the current time, the prerequisite activity of self annotation is in full swing on the current Web, thanks to the inflective effects of Web 2.0.

BTW - Would this URI to all Semantic Web related posts on my blog pass the Web 2.0 litmus test? Likewise, this URI to all Web 2.0 related posts? I wonder :-)

Microsoft Gadgets, Start.com and Innovation: "

A lot of the comments in the initial post on the Microsoft Gadgets blog are complaints that the Microsoft is copying ideas from Apple's dashboard. First of all, people should give credit where it is due and acknowledge that Konfabulator is the real pioneer when it comes to desktop widgets. More importantly, the core ideas in Microsoft Gadgets were pioneered by Microsoft not Apple or Konfabulator.

From the post A Brief History of Windows Sidebar by Sean Alexander

Microsoft 'Sideshow*' Research Project (2000-2001)

While work started prior, in September 2001, a team of Microsoft researchers published a paper entitled, 'Sideshow: Providing peripheral awareness of important information' including findings of their project.
...
The research paper provides screenshots that bear a striking resemblance to the Windows Sidebar. The paper is a good read for anyone thinking about Gadget development. For folks who have visited Microsoft campuses, you may recall the posters in elevator hallways and Sidebar running on many employees desktops. Technically one of the first teams to implement this concept

*Internal code-name, not directly related to the official, ‘Windows SideShow™’ auxiliary display feature in Windows Vista.>

Microsoft ‘Longhorn’ Alpha Release (2003)

In 2003, Microsoft unveiled a new feature called, 'Sidebar' at the Microsoft Professional Developer’s Conference. This feature took the best concepts from Microsoft Research and applied them to a new platform code-named, 'Avalon', now formally known as Windows Presentation Foundation...

Microsoft Windows Vista PDC Release (2005)

While removed from public eye during the Longhorn plan change in 2004, a small team was formed to continue to incubate Windows Sidebar as a concept, dating back to its roots in 2000/2001 as a research exercise. Now Windows Sidebar will be a feature of Windows Vista. Feedback from customers and hardware industry dynamics are being taken into account, particularly adding support for DHTML-based Gadgets to support a broader range of developer and designer, enhanced security infrastructure, and better support for Widescreen (16:10, 16:9) displays. Additionally a new feature in Windows Sidebar is support for hosting of Web Gadgets which can be hosted on sites such as Start.com or run locally. Gadgets that run on the Windows desktop will also be available for Windows XP customers – more details to be shared here in the future.

So the desktop version of 'Microsoft Gadgets' is the shipping version of Microsoft Research's 'Sideshow' project. Since the research paper was published a number of parties have shipped products inspired by that research including MSN Dashboard, Google Desktop and Desktop Sidebar but this doesn't change the fact that the Microsoft is the pioneer in this space.

From the post Gadgets and Start.com by Sanaz Ahari

Start.com was initially released on February 2005, on start.com/1 – since then we’ve been innovating regularly (start.com/2, start.com/3, start.com and start.com/pdc) working towards accomplishing our goals:

• To bring the web’s content to users through:
  • Rich DHTML components (Gadgets)
  • RSS and behaviors associated with RSS
  • High customizability and personalization
• To enable developers to extend their start experience by building their own Gadgets

Yesterday marked a humble yet significant milestone for us – we opened our 'Atlas' framework enabling developers to extend their start.com experience. You can read more it here: http://start.com/developer. The key differentiators about our Gadgets are:

• Most web applications were designed as closed systems rather than as a web platform. For example, most customizable 'aggregator' web-sites consume feeds and provide a fair amount of layout customization. However, the systems were not extensible by developers. With start.com, the experience is now an integrated and extensible application platform.
• We will be enriching the gadgets experience even further, enabling these gadgets to seamlessly work on Windows Sidebar

The Start.com stuff is really cool. Currently with traditional portal sites like MyMSN or MyYahoo, I can customize my data sources by subscribing to RSS feeds but not how they look. Instead all my RSS feeds always look like a list of headlines. These portal sites usually use different widgets for display richer data like stock quotes or weather reports but there is no way for me to subscribe to a stock quote or weather report feed and have it look the same as the one provided by the site. Start.com fundamentally changes this model by turning it on its head. I can create a custom RSS feed and specify how it should render in Start.com using JavaScript which basically makes it a Start.com gadget, no different from the default ones provided by the site.

From my perspective, we're shipping really innovative stuff but because of branding that has attempted to cash in on the 'widgets' hype, we end up looking like followers and copycats.

Marketing sucks.

(Via Dare Obasanjo aka Carnage4Life.)

I ran into two work-related problems today that left me feeling like there are some aspects of two very recent (Web 2.0-esque if we wish to join the buzzword orgy of late) architectures (REST/Services and XForms) that are problematic:

Demonstrating an Achilles Heel Of XML Messaging

XML as a medium for remote communication (evangelized more with WSDL-related architectures than in REST) has over-stated its usefullness in at least one concrete regard, in my estimation. I've had a hard time taking most of the architectural arguments on the pros/cons of SOAP/XML-RPC versus REST seriously because it seems to be nothing more than buzzword warfare. However, I recently came across a concrete, real world example of the pitfalls of implementing certain remote service needs on XML-based communication mediums (such as SOAP/XML-RPC).

If the objects/resources you wish to manipulate at the service endpoints are run of the mill (consider the standard cliche purchase order example(s)) then the benefits of communicating in XML is obvious: portability, machine readability, extensibility, etc.. However consider the scenario (which I face) in which the objects/resources you wish to manipulate are XML documents themselves! This scenario seems to work to the disadvantage of the communication architecture.

Lets say you have a repository at one end (which I do) that has XML documents you wish to manipulate remotely. How do you update the documents? I've discussed this before (see: Base64 encoded XML content from an XForm) so I'll spare the details of the problem. However, I will mention that in retrospect this particular problem further emphasizes the advantage of a MinimalistRemoteProcedureCall (MRPC) approach - MRPC is my alternative acronym for REST :).

Consider the setContent message:

[SOAP:Envelope]
    [SOAP:Body]
     [foo:setContent]
       [path] .. path to document [/path]
       [src]... new document as a fragment ...[/src]
     [/foo:setContent]
   [/SOAP:Body]
[/SOAP:Envelope]

Notice that the location of the resource we wish to update is embedded within the message transmitted (via SOAP), which is transported on top of another communication medium (HTTP) that already has the neccessary semantics for saying the same thing:

Set the content of the resource identified by a path

In the SOAP scenario, the above message is delivered to a single service endpoint (which serves as an external gateway for all SOAP messages) which has to then parse the entire XML message in order to determine the method invoked (setContent in this case) and the parameters passed to it (both of which are only header information on a document that consists mostly of the new document).

However, in the MRPC scenario this service would be invoked simply as an HTTP PUT request sent directly to the XML document we wish to update:

Method: PUT
Protocol:  HTTP/1.0
URI: http://remoteHost:port/< .. path to XML document ..>
CONTENT:
... new document in it's entirety ..

Here, there is no need for a service middleman to interpret the service requested (and no need to parse a large XML document that contains another document embedded as a fragment). The HTTP request by itself specifies everything we need and does it using HTTP alone as the communication medium. This is even more advantageous when the endpoint is a repository that has a very well defined URI scheme or general addressing mechanism for it's resources (which 4Suite does, the repository in my case).

The Headaches of Base 64 Encoding in XForms

Since i didn't have the option of a REST-based service architecture (the preferred solution) I was relegated to having to base64 encode the new XML content and embed it within the XML message submitted to the service endpoint, like so:

[SOAP:Envelope]
   [SOAP:Body]
     [foo:setContent]
       [path] .. path to document [/path]
       [src]... base64 encoding of new document's serialization ...[/src]
     [/foo:setContent]
   [/SOAP:Body]
[/SOAP:Envelope]

Base 64 seemed like the obvious encoding mechanism mostly because it would seem from an interpretation of the XForms specification that due to the data binding restrictions of the Upload Control when bound to instances of type xsd:base64Binary a conforming XForms processor is responsible for having the capability to encode to Base 64 on the fly. Now, this is fine and dandy if the XML content you wish to submit is retrieved from a file on the local file system of the client communicating remotely with the server. However, what if you wish to use an instance (a live DOM) as the source for the update? This seems like a very reasonable requirement given that one of the primary motivation of XForms is to encourage the use of XML instances as the user interface data model (providing a complete solution to the 'M' in the MVC architecture.)

However:

• There is no mechanism within XForms for serialising live instances (there needs to be such a standard so implementations don't create their own proprietary mechanisms)
• There is no mechanism within XForms for explicitely encoding text in some portable binary format (which is incredibly useful IMHO - as shown above)

(Via Uche Ogbuji.)

Piggy Bank 2.0 Beta

New release of Piggy Bank, the Semantic Web extension for Firefox. It harvests data as you browse (when you click a status bar indicator), which can later be searched and viewed in a facetted browser.

The docs have come along some too -

Piggy Bank can collect pure information in the following cases:

1. The web page has invisible link(s) to RDF data (encoded in RDF/XML or N3 formats).
2. The web page exports an RSS feeds.
3. The address of the web page is a file:/ URL pointing to a directory.
4. Piggy Bank has a "screen scraper"€ [XSLT or Javascript] that can re-structure the web page HTML code into RDF data.

There's a tutorial on writing Javascript screenscrapers on the site, nice touch.

I have also added an architecture diagram to accelerate comprehension (a picture speaks a thousand words...):

Ajax, Hard Facts, Brass Tacks ... and Bad Slacks

A number of people have contacted me recently about Ajax [1] -- a catchy name -- coined to provide an umbrella term for a particular group of technologies used to build web applications. The use of the word comes from Jesse James Garrett in a recent blog [2], and describes a class of internet applications written using JavaScript in a browser. By using JavaScript these apps have full access to the DOM, and as a consequence are able to make all sorts of changes to the page that the user is interacting with, without having to go back to the server.

When the application does need to go back to the server -- to deliver some data and get a response -- the idea is to keep the DOM intact so that the user has a smooth experience. This means that all communication with the server needs to take place outside of the normal HTML form mechanism, since this would obviously replace the current page.

Ajax addressed this, with what it calls 'asynchronous-JavaScript' -- retrieve only the data you need, and then directly manipulate the DOM to get the effect you want. 'Asynchronous-JavaScript' accounts for the first few letters of the name, with the remainder being the obligatory 'X' for XML (although XML is not really key to this technology, and many of the applications that are often cited as Ajax-apps don't use XML as the data medium).

Buzzing

The response to Ajax has been pretty positive. In fact the only negatives have been either to suggest a change of name or to moan a little that "I've been doing this for years, why hasn't anyone noticed me?" (I won't put any links to those sort of articles, since they are a little embarassing -- after all, everyone has been doing this for years!)

Anyway, despite a couple of sour-pusses, the software community is almost universally excited, and the blog wires have glowed over the last few months with descriptions of Google Maps, GMail, and so on.

Just about everyone who has asked me about Ajax has expected me to be disappointed. Surely, they say, this makes the case for XForms weaker? But my answer is the exact opposite -- XForms and standards-based web applications are in every way superior to the techniques described as Ajax, since the whole raison d'être of XForms and XHTML 2 is to address the very problems that Ajax-like techniques suffer from.

That may come across as a little bold...so perhaps I should explain.

From Workaround to Feature

We've all been using HTML mark-up for years now, and the language hasn't changed much in that time. As a consequence, the increasing demand for more complex web-pages has meant that the balance in our documents has shifted increasingly from vanilla mark-up to 'the workaround'.

Whether it's providing tooltips, dynamic/repeating data sections, or small portions of our page that change without having to request a new document, we've generally had to dive into script. But the shift from mark-up to script has meant that the mark-up language itself has been relegated to a mere carrier for programs.

Unfortunately this means that no-one gains -- it's annoying for the programmer to have to produce ever more convoluted spaghetti JavaScript to meet the demands of their audience, but it's also annoying for the non-programmer, who probably only wants a tooltip. And its particularly annoying for those who want to use documents on the web for more ambitious applications to find that most of the important stuff in a document is hidden away in script.

All is not lost, however, since this collection of 'workarounds' provides a rich source of real-life patterns that appear for authors and programmers, time and again. They may be workarounds, but they are much-needed ones.

The aim of the new generation of languages like XForms and XHTML 2 is to take these 'common patterns' and turn them into mark-up. Just like the HTML elements <a> and <form> pack an enormous amount of functionality into deceptively simple tags, so too can new declarative mark-up capture patterns that have emerged 'in the wild'.

(Note that this is the opposite of so-called folksonomies, where popular practice that occurs in the wild is left it the wild, and codification is regarded as a dirty word.)

The XML HTTP Request Object

Let's take the much talked about XML HTTP Request Object (XMLHttpRequest). If you are not familiar with it, it was originally part of Microsoft's XML parser, and allows you to send and receive data outside of the normal HTML form processing. Since it's a handy feature to have in a client, other browsers have followed suit and it's now becoming the 'standard' way to communicate with servers without messing up your page. It's a corner-stone of Ajax. (A good summary with examples is on Jim Ley's jibbering.com site [3].)

But...we need to be clear that we're using XMLHttpRequest to get round a weakness in HTML forms. The problem we have is that even if you know that a server is about to give you some data, and the server knows it's about to give you some data, there's no way to tell your form that -- instead your page will be wiped out and replaced with whatever the server sends back.

Of course, constant round-tripping doesn't make it completely impossible to produce applications, and a lot of books and airline tickets are bought every day without the facility to get 'just the data'. But we all know it would reduce network traffic and create a smoother user experience if we could just send a list of books or seats, rather than a whole new page.

Over the years applications such as Microsoft's Outlook Web Access (OWA), have had to step around the HTML form to get just the data they need. But, whilst OWA considerably predates GMail, until the advent of XMLHttpRequest, the techniques used were quite difficult to manage. (Google Suggest is often cited as a good example of an Ajax-app, but interestingly merges old and new techniques; XMLHttpRequest is used to obtain a piece of JavaScript from a server, and this script contains a call to a client-side function, but using server-provided parameters. It's one of the techniques you might have used in the past with a hidden frame.)

So as many have said on their blogs, XMLHttpRequest is not a newly devised technique, but rather a generally accepted replacement for a very old technique. But ultimately that technique is a workaround since the real problem is that HTML forms will always replace the current page.

Beyond HTML Forms

Whilst XMLHttpRequest gives us a way to get data to and from the server without losing our document, we've unfortunately thrown the baby out with the bath-water; whatever the weaknesses of HTML forms, you have to acknowledge that they are pretty simple to use. Here's an abbreviated version of Google's search form (note that the mark-up is HTML, not XML):

<form action=/search name=f>
  <input type=hidden name=hl value=en>
  <input maxLength=256 size=55 name=q value="">
  <input type=submit value="Google Search" name=btnG>
</form>

As you can see, the simple problem with HTML forms is that we don't say anything about where the data should go when we've received it from the server. The assumption in HTML of old is that we are just doing a kind of 'super-navigation', and no matter what we send to the server, it will only ever give us back a new web-page. (To put it a different way, you could say that <a> and <form> are pretty much the same thing.)

To see how this problem is resolved, let's code the same Google search in XForms:

<xf:submission id="sub-search"
 action="http://www.google.com/complete/search?hl=en"
 method="get" separator="&"
 replace="all"
/>


<xf:input ref="q">
  <xf:label>Query:</xf:label>
</xf:input>


<xf:submit submission="sub-search">
  <xf:label>Google Search</xf:label>
</xf:submit>

Although it will do exactly the same -- right down to replacing the current page -- it's a little different to the HTML mark-up. But the changes in structure have given us some major benefits, from accessible labels on our form controls, to the possibility of many different submissions for the same data.

But what it has also given us is the possibility of solving our data update problem. The replace attribute is actually optional in XForms, but I showed it in the previous mark-up so that you can compare it to this:

<xf:submission id="sub-search"
 action="http://www.google.com/complete/search?hl=en"
 method="get" separator="&"
 replace="instance"
/>

In this example the data returned from the server will just replace the instance that was sent, and our page will remain completely intact. (The replace attribute can take the values all, instance, or none.)

I won't show the full equivalent using XMLHttpRequest since it's pretty large, but I'll give a flavour of it. (Jim Ley's page -- referenced earlier -- shows how to search Google with XMLHttpRequest.)

The Script Version

First we need to create an XMLHttpRequest object, but we need to do it in such a way that it will work on both Mozilla and IE:

var req;


function loadXMLDoc(url) {
    // native XMLHttpRequest object
    if (window.XMLHttpRequest) {
        req = new XMLHttpRequest();
        req.onreadystatechange = readyStateChange;
        req.open("GET", url, true);
        req.send(null);
    // IE/Windows ActiveX version
    } else if (window.ActiveXObject) {
        req = new ActiveXObject("Microsoft.XMLHTTP");
        if (req) {
            req.onreadystatechange = readyStateChange;
            req.open("GET", url, true);
            req.send();
        }
    }
}

When a document is loaded via this function, the readyStateChange() method is invoked:

function readyStateChange() {
    // '4' means document "loaded"
    if (req.readyState == 4) {
        // 200 means "OK"
        if (req.status == 200) {
            // do something here
        } else {
            // error processing here
        }
    }
}

From a programming point of view, I guess you could say that there isn't a lot wrong with this, but then from a programming point of view there wasn't a lot wrong with Z80 or 6502 assembly languages -- I just wouldn't want to go back to them!

But the most important issue is that we have lost the very thing that was responsible for HTML's success -- the use of simple, clear, declarative mark-up, in which we simply state our intent, without having to write a program to do it for us. After all, the web took off because authors only had to master <a> in order to enter the exciting new world of 'hypertext' -- but XMLHttpRequest raises the bar again, and takes us right back into the heart of geek-world.

Beyond XMLHttpRequest

But in keeping with the principle that I outlined above -- that XForms and XHTML 2 try to provide mark-up for commonly existing design patterns -- let's see if there are any other patterns that XMLHttpRequest has thrown up.

You will have noticed in the earlier script that we had tests for success and failure:

if (req.status == 200) {
  // do something here
} else {
  // error processing here
}

XForms provides the same functionality through the use of events -- on success do this, on failure do that. This is far more powerful, since it hides the protocol-specific aspects of this code ("200" may be 'success' for HTTP, but it isn't 'success' when saving data to the hard-drive or sending an email).

XForms uses declarative mark-up to express those events, which again dramatically reduces coding:

<xf:action ev:observer="sub-search" ev:event="xforms-submit-error">
  <xf:message level="modal">
    Submission failed
  </xf:message>
</xf:action>

But there's lots, lots more in the submission part of XForms:

• it can provide full XML Schema validation before submitting the data;


• there is built in support for numerous types of serialisation, such as multipart/related;


• abstract methods are used so the code is independent of protocol. For example, since put means the same thing whether the target URL begins http: or file:, a form with relative paths will run unchanged on a local machine or a web server;


• it's extensible -- in formsPlayer 2.0 we have used the submission element to read and write from an ADO database, allowing programmers to convert forms from using the web to using a local database by doing nothing more than changing a single target URL. (Try doing that with XMLHttpRequest!)

The submission part of XForms is in fact so powerful that it will eventually form a separate specification, for use in other languages.

From Patterns to Mark-up

And there are plenty more patterns out there that were crying out to be turned into mark-up, and which are now incorporated into XForms and XHTML 2. Do you remember the days when if we wanted a tooltip that contained mark-up -- perhaps an image, or bold text -- we had to use a carefully placed <div>, a CSS display: none;, a mouseover event handler and a timer? Nowadays the programmer with better things to do than work with spaghetti-JavaScript just uses the XForms <hint> element, and for free they get platform independence (and therefore accessibility), as well as the ability to insert any mark-up.

And what about the days when we had to write code to open up a text-to-speech engine, and then invoke the various methods on the object to get it to speak its mind? Nowadays who wouldn't just use a CSS property on their XForms' messages?

Bad Slacks

And do you remember...I'm sorry, this one always makes me laugh...do you remember how we used to write lots of JavaScript to recalculate the shopping-cart when a new item was added? I know it's hard to believe -- it's like looking at old photos of us all wearing flares. Anyway, thank God for straight trousers and the XForms dependency-engine.


But enough of the good old days, the days of assembly language, C and JavaScript...let's stick with the new.

Do Try This at Home

To round all of this off, we'll take a look at Google Suggest, and we'll use XForms to implement it. I'll walk through the demo in a separate blog [4] so that this one doesn't get too cluttered -- and hopefully by disecting this simple but useful application, we can show how declarative mark-up scores over scripting.


[1] Will AJAX help Google clean up?, c|net, http://news.com.com/Will+AJAX+help+Google+clean+up/2100-1032_3-5621010.html

[2] Ajax: A New Approach to Web Applications, Jesse James Garrett, Adaptive Path blog, http://www.adaptivepath.com/publications/essays/archives/000385.php

[3] Using the XML HTTP Request object, http://jibbering.com/2002/4/httprequest.html

[4] "Google Suggest" Using XForms, http://internet-apps.blogspot.com/2005/04/google-suggest-using-xforms.html

Tags: xforms | xbl | webapps | ajax | javascript

Advertising in RSS is just starting now, for all practical purposes. If we wanted to, as an industry, reject the idea, we could.

Here goes:

Blog Editing

I can use any editor that supports the following Blog Post APIs:

- Moveable Type

- Meta Weblog

- Blogger

Typically I use Virtuoso (which has an unreleased WYSIWYG blog post editor), Newzcrawler, ecto, Zempt, or w.bloggar for my posts. If a post is of interest to me, or relevant to our company or customers I tend to perform one of the following tasks:

- Generate a post using the "Blog This" feature of my blog editor

- Write a new post that was triggered by a previously read post etc.

Either way, the posts end up in our company wide blog server that is Virtuoso based (more about this below). The internal blog server automatically categorizes my blog posts, and automagically determines which posts to upstream to other public blogs that I author (e.g http://kidehen.typepad.com ) or co-author (e.g http://www.openlinksw.com/weblogs/uda and http://www.openlinksw.com/weblogs/virtuoso ). I write once and my posts are dispatched conditionally to multiple outlets.

RSS/Atom/RDF Aggregation & Reading

I discover, subscribe to, and view blog feeds using Newzcrawler (primarily), and from time to time for experimentation and evaluation purposes I use RSS Bandit, FeedDemon, and Bloglines. I am in the process of moving this activity over to Virtuoso completely due to the large number of feeds that I consume on a daily basis (scalability is a bit of a problem with current aggregators).

Blog Publishing

When you visit my blog you are experiencing the  soon to be released Virtuoso Blog Publishing engine first hand, which is how WebDAV, SQLX, XQuery/XPath, and Free Text etc. come into the mix.

Each time I create a post internally, or subscribe to an external feed, the data ends up in Virtuoso's SQL Engine (this is how we handle some of the obvious scalability challenges associated with large subscription counts). This engine is SQL2000N based, which implies that it can transform SQL to XML on the fly using recent extensions to SQL in the form of SQLX (prior to the emergence of this standard we used the FOR XML SQL syntax extensions for the same result). It also has its own in-built XSLT processor (DB Engine resident), and validating XML parser (with support for XML Schema).  Thus, my RSS/RDF/Atom archives, FOAF, BlogRoll, OPML, and OCS blog syndication gems are all live examples of SQLX documents that leverage Virtuoso's WebDAV engine for exposure to Blog Clients.

Blog Search

When you search for blog posts using the basic or advanced search features of my blog, you end up interacting with one of the following methods of querying data hosted in Virtuoso: Free Text Search, XPath, or XQuery. The result sets produced by the search feature uses SQLX to produce subscription gems (RSS/Atom/RDF/OpenSearch) and URIs that enable dynamic tracking of my posts using your search keywords.

BTW - the http://www.openlinksw.com/blog/~kidehen blog home page exists as a result of Virtuoso's Virtual Domain / Multi-Homing Web Server functionality. The entire site resides in an Object Relational DBMS, and I can take my DB file across Windows, Solaris, Linux, Mac OS X, FreeBSD, AIX, HP-UX, IRIX, and SCO UnixWare without missing a single beat! All I have to do is instantiate my Virtuoso server and my weblog is live.

BTW - I took the time to update my public blog-he-roll and new blog-her-roll; both being tiny snapshots of my actual blog subscription collection, which by the way, is actually so large and diverse that it's part of an internal project covering distributed XQuery and scalability :-)

Here is a an interesting and insightful quote from the article that resonates with me:

Invisibility is an accidental and little-recognized characteristic of digital knowledge work. Seeing the problem is the first step to a solution. While better technology tools will play an important role, the next steps are changes in attitude and behavior at the individual and work group level. For example, organizing your own digital files into project-related directories can help, but not if you continue to name files "FinalPresentationNN.doc" where NN is some number between 1 and 15 representing a crude effort at version control. Embed more information in the file name where you know it will be visible even as you e-mail it around the organization. Use more informative subject lines on your email. Those file names and subject lines should provide the best clues possible as to what will be found inside.

The quote above strikes a chord with me because I have spent a majority of my professional career working on technology that is aimed at Information and Knowledge workers. It also has uncanny timing as it sheds light on a major aspect of the next major release of Virtuoso that aims to continue the process of unveiling the intrinsic value of Unified Storage (SQL, XML, and Multimedia content) for Knowledge workers.

We are already experiencing a rapid build up of XML content and binary data with XML based metadata annotations as a result of the network effects of the Blogosphere and Wikisphere. This content explosion ultimately provides context for understanding the value of URIs association with collections of physically (e.g hierarchical directory structure) or logically (Tagging or Dynamic Filtering) partitioned content. 

To be continued..

The Internet Archive initiative is building up an amazing collection of content that includes this "must watch" movie about the somewhat forgotten hypercard development environment.

As I watched the hypercard movie I obtained clear reassurance that my vision of Web 2.0 as critical infrastructure for a future Semantic Web isn't unfounded. The solution building methodology espoused by hypercard is exactly how Semantic Web applications will be built, and this will be done by orchestrating the componentary of Web 2.0.

When watching this clip make the following mental adjustments:

1. Swap hypercard stacks for discrete and/or composite services that have published endpoints exposed by Web 2.0 points of presence
   
   
2. Think of information taking the form of XML based content e.g. RSS, Atom, RDF, FOAF, XFN, and other future XML based data contextualization formats; all accessible via URIs
   
   
3. When the Apple Mac operating system is mentioned (or infered) think of the Internet (you don't need Windows, Mac OS, Linux, UNIX etc. to realize the vision, the network provided by the Internet is the Operating System)
   
   
4. When the Apple computer is mentioned simply think about a plethora of function specific devices (computers, mobile phones, PDAs etc.) that overtly or covertly provide conduits to the new operating environment (the Internet)
   
   
5. As you hear term "whole new body of people that are non programmers contributing there ideas" think about yourself and the increasing ease of participation that's beginning to take shape in this emerging frontier!
   
   
6. As for "Whole Earth Catalog", think Wikipedia or more recent efforts such as Answers.com.

Web 2.0 is a reflection of the web taking its first major step out of the technology stone age (certainly the case relative to the hypercard movie and "pre web" application development in general).

Udell to event promoters on leveraging folksonomy: 'Pick a tag' I'm now trying to figure out why InfoWorld's Jon Udell is a journalist and not a millionaire technologist (or maybe he is). Udell keeps coming up with one brilliant idea after another. The first of these -- which I thought was just plain obvious -- was Udell's idea for vendors ...

I do know Jon (albeit primarily via emails and phone interviews), he even put me forward for an innovators award in 2003 re. Virtuoso etc.

Friendster befriends blogs--and fees Two Web trends converge as the social networking site prepares to launch blogs through partnership with Six Apart.

Today is one of those days where one topic appears to be on the mind of many across cyberspace. You guessed right! Its that Web 2.0 thing again.  

Paul Bausch brings Yahoo!'s most recent Web 2.0 contribution to our broader attention in this excerpt from his O'Reilly Network article:

I browse news, check stock prices, and get movie times with Yahoo! Even though I interact with Yahoo! technology on a regular basis, I've never thought of Yahoo! as a technology company. Now that Yahoo! has released a Web Services interface, my perception of them is changing. Suddenly having programmatic access to a good portion of their data has me seeing Yahoo! through the eyes of a developer rather than a user.

The great thing about this move by Yahoo! is two fold (IMHO):

1. It certainly makes Yahoo! a little more interesting of late. And it will certainly helps to distinguish Yahoo! from Google. Of course these companies overlap somewhat, but they are also pretty different in focus. I see Yahoo! increasingly as a portal platform play providing content access via syndication, publishing, and web services.
   
   
2. It will impact their bottom line pretty rapidly, and I hope they realize the impact of Web 2.0 when trying to explain the growth increments whenever they next report to their investors :-) In a previous post I expressed my sense of some confusion on the part of Jeff Bezos regarding the total contribution of AWS to Amazon's growth (BTW - my articles to date re. Amazon and Web 2.0 are available from here in a variety of XML syndication formats: Atom, RSS 2.0, RDF).
   

The great thing about the Platform oriented Web 2.0 is the ability to syndicate your value proposition (aka products and services) instead of pursuing fallable email campaigns. It enables the auto-discovery of products and services by user agents (the content aspect). Web 2.0 also provides an infrastructure for user agents to enter into a consumptive interactions with discrete or composite Web Services via published endpoints exposed by a platform (the execution aspect).

A scenario example:

You can obtain RSS feeds (electronic product catalogs) from Amazon today, although you have to explicitly locate these catalog-feeds since Amazon doesn't exploit feed auto-discovery within their domain.

If you use Firefox or another auto-discovery supporting RSS/Atom/RDF user agent; visit this URL; Firefox users should simply click on the little orange icon bottom right of the browser's window to its RSS feed auto-discovery in action.

Anyway, once you have the feeds the next step is execution endpoints discovery within the Amazon domain (the conduits to Amazon's order processing system in this example). At the current time there isn't broad standardization of Web Services auto-discovery but it's certainly coming; WSIL is a potential front runner for small scale discovery while UDDI provides a heavier duty equivalent for larger scale tasks that includes discovery and other related functionality realms.

Back to the example trail, by having the RSS/Atom/RDF feed data within the confines of a user agent (an Internet Application to be precise) nothing stops the extraction of key purchasing data from these feeds, plus your consumer data en route to assembling an execution message (as prescribed by the schema of the service in question)for Amazon's order processing/ shopping cart service.  All of this happens without ever seeing/eye-balling the Amazon site (a prerequisite of Web 1.0 hence the dated term: Web Site).

To summarize: Web 2.0 enables you to syndicate your value proposition and then have it consumed via Web Services, leveraging computer, as opposed to human interaction cycles. This is how I believe Web 2.0 will ultimately impact the growth rates (in most cases exponentially) of those companies that comprehend its potential. 

It is clear that in comparison to the Web of the last century, the nature of data on the Web later in this decade will be very different in the following aspects:

• Volume of data is growing by orders of magnitudes every year
  Multimedia and sensor data are becoming more and more common.
  
  
• Spatio-temporal attributes of data are important.
  
  
• Different data sources provide information to form the holistic picture.
  
  
• Users are not concerned with the location of data source, as long as its quality and credibility is assured. They want to know the result of the data assimilation (the big picture of the event).
  
  
• Real-time data processing is the only way to extract meaningful information
  Exploration, not querying, is the predominant mode of interaction, which makes context and state critical.
  
  
• The user is interested in experience and information, independent of the medium and the source.
  

Effectively, the nature of the knowledge on the Web is changing very fast. It used to be mostly static text documents; now it will be a combination of live and static multimedia, including text, data and documents with spatio-temporal attributes. Considering these changes, can the search engines developed for static text documents be able to deal with the needs of the Web? [via E M E R G I C . o r g]

No, but this doesn't render them useless since we wouldn't be at this point without the likes of Google, Yahoo! et al. But building upon the data substrate that web data oriented search engines provide is where the next batch of Information access and Knowledge discovery solutions will carve out their space. The symbiotic relationship between Google (data) and Gurunet's Answers.com (Information and Knowledge) is one interesting example.

The Web is a distributed collection of databases that implement variety of data storage models but are commonly accessible via protocols that rely on HTTP for transport (in-bound and out-bound messages) services. These databases increasingly using well-formed XML for query result (data contextualization) persistence and URIs for permenant reference. 'What Database?" you might ask, "What you once called your Web Site, Blog, Wiki, etc.." my time-less reply.

When you have the database that I describe above, and a collection of entry points from which discrete or composite Web Services can be invoked available from one or more internet domains, you end up with what I prefer to call "Web 2.0" presence, or what Richard McManus describes as: "The Web as a Platform".

Here is a collection of posts I have made in the past relating to Web 2.0, note that this list is dynamic since this blog is Virtuoso based (predictably):

Free Text Search with XHTML results page (with Virtuoso generated URIs for RSS, Atom, and RDF): http://www.openlinksw.com/blog/search.vspx?blogid=127&q=web+2.0&type=text&output=html 

It's also no secret that I believe that Virtuoso is a bleeding edge Web 2.0 technology platform (and more..). The URIs that I am exposing provide the foundation layer for other complimentary Web initiatives such as the Semantic Web (Web 2.0 provides infrastructure for the Semantic Web as time will show). They are also completely usable outside the realm of this blog.

BTW - Jon Udell is writing, experimenting with, and demonstrating similar concepts across feeds within his Web 2.0 domain.

These are indeed fun times!

Have RSS feeds killed the email star? silicon.com Feb 28 2005 12:58PM GMT

Anyway, back to cognitive dissonance. Could this be the reason for the following?

1. Open Source products are increasingly database specific even though they could be database independent via Open Source ODBC SDK efforts such as iODBC and unixODBC. We increasingly narrowing our choices down to database specific "Closed Source" or database specific "Open Source" solutions and somehow deem this to be progress
   
2. The prevalent use of free standards compliant data access drivers (ODBC, JDBC, and ADO.NET) or their native counterparts that remain vulnerable to simple password hacks (there are databases behind those dynamic web sites!!) as none of these have any notion of "rules based" authentication and data access policy
   
3. The time-tested fallacy that: "select * from table" defines a viable RDBMS engine since Transaction Atomicity, Concurrency, Isolation, and Durability (ACID) mean zip! Ditto scrollable cursors, stored procedures, and other presumably useless aspects of any marginably decent RDBMS engine
   
4. Failing to comprehend that a Weblog is your property (if you have a personal blog) not the property of the vendor hosting your service (that important issue of separating data ownership and data storage again). You may have heard about, or experienced, total loss of weblog and/or weblog archives arising from weblog engine or blog service provider changeovers
   
5. Failing to see the synergy between personal/group/corporate information stores (aka infobase) such as Wikis, Weblogs, and the burgeoning semantic web. Jon Udell for instance, is trying to get the point across via his tireless collection of XQuery/XPath based queries aimed at the blogosphere section of the burgeoning semantic web. Here are some of mine (scoped to this weblog):
   
• Security related posts to date (XPath query)
  
• Infobase related posts to date (Free Text search)
  

And more...

By Uche Ogbuji, IBM developerWorks

The world of XML and Web services is huge, and growing. developerWorks does much to map it out for you, but when you're looking for a schema or a public Web service to meet some pressing need, it's useful to have handy several key resources. This tip shows you how to comb through the enormous variety of Internet resources to find schemata and Web services using common search criteria. The best known source for finding public SOAP Web services is XMethods. It has a comprehensive list of SOAP services that you can sort by several criteria. It also provides a demo client so you can try out the services right from the index site. You can also keep track of the listings on XMethods programmatically using UDDI, RSS, and other means.sites that provide directories of Web services include RemoteMethods.com and Web Service List. A chronicle of interesting Web services is Web service of the Day.

One resource that straddles the Web services/Semantic Web is WSindex.org, a directory of Web services, XML, SOAP, UDDI, WSDL, and Semantic Web resources. This site is a hierarchical and searchable directory.

http://www-106.ibm.com/developerworks/xml/library/x-tiplkws.html

Email As A Platform It looks like more people are starting to realize that email is more than it seems. Especially given the drastic increase in storage size of web-based email applications, more people are realizing that email is basically a personal database. People simply store information in their email, from contact information that was emailed to them to schedule information to purchase tracking from emailed receipts. Lots of people email messages to themselves, realizing that email is basically the best "permanent" filing system they have. That's part of the reason why good email search is so important. Of course, what the article doesn't discuss is the next stage of this evolution. If you have a database of important information, the next step is to build useful applications on top of it. In other words, people are starting to realize that email, itself, is a platform for personal information management.

By Bill Gates, Microsoft Executive Mail

Microsoft's product interoperability strategy: "First, we continue to support customers' needs for software that works well with what they have today. Second, we are working with the industry to define a new generation of software and Web services based on eXtensible Markup Language (XML), which enables software to efficiently share information and opens the door to a greater degree of 'interoperability by design' across many different kinds of software. Our goal is to harness all the power inherent in modern (and not so modern) business software, and enable them to work together so that the whole is greater than the sum of the parts. We want to further eliminate friction among heterogeneous architectures and applications without compromising their distinctive underlying capabilities... The XML-based architecture for Web services, known as WS-* ('WS-Star'), is being developed in close collaboration with dozens of other companies in the industry including IBM, Sun, Oracle and BEA. This standard set of protocols significantly reduces the cost and complexity of connecting disparate systems, and it enables interoperability not just within the four walls of an organization, but also across the globe."

http://www.microsoft.com/mscorp/execmail/2005/02-03interoperability.asp

Amen Bill! As long as this doesn't covertly imply "Windows Specificity" by way of "Interoperability" becoming a "Windows Unique Selling Point"!

As per usual, the devil will be in the implementation details of your company's products.

By Martin LaMonica, CNET News.com

The World Wide Web consortium, the standards body in charge of developing XML, said Tuesday that it has issued three recommendations designed to make handling XML-formatted data more efficient. The specifications have the backing of large industry software providers, including IBM, Microsoft and BEA Systems, which provide the software infrastructure to build and run XML data and Web services applications.

The W3C and vendors are looking at a variety of methods of speeding up the performance of XML, which can be slow for certain applications.

http://news.com.com/2110-1013_3-5551788.html

See also the news story: http://xml.coverpages.org/ni2005-01-25-a.html

By Peter Sefton, XML.org

The author explores some of the ways that OpenOffice.org's Writer application is open to customization and configuration. He coveres a few techniques that will be of interest to template maintainers working with OpenOffice.org writer: how to crack open the file format, how to maintain large sets of styles, and how to customize menus and macros, all without using anything except standard tools, zip, an XSLT processor, and a text editor. All this can, of course, be further automated with a programming language of some kind, even a batch file.

There are some changes coming in version 2 of OpenOffice.org, but all these techniques will be forwards compatible, although some things like the location and name of the menu-bar files look like they will change. If you are also trying to store and manipulate content in XML but want to use a word processing environment for authoring, then well-crafted templates are even more important.

http://www.xml.com/pub/a/2005/01/26/hacking-ooo.html

See also the OpenDocument 1.0 CD: http://xml.coverpages.org/ni2005-01-04-a.html

1. What is this standard
   
2. Why is it important
   
3. What are the implementation challenges

When XQuery first came across my radar (late 90s even before "XQuery" became the moniker for an XML Query Language) I arrived at the following conclusions using the steps listed above:

1. What is XQuery about? Its about querying XML Documents (at the time real or virtual) in a repository. Basically, its the SQL equivalent for the XML based Infobase;
   
2. Why is it important? Because we will need to access, repurpose, and disseminate the contents of the Infobase for a myriad of reasons which ultimately culminate in knowledge creation;
   
3. What are the implementation challenges? Where do I start? Anway, here are a few:
   
• Content Creation - we need to create the Infobase; for an XML based Infobase

As indicated in an earlier post: IBM is clearly validating what we have done with Virtuoso (as was the case initially with their Virtual / Federated DBMS initiative ala DB2 Integrator). Here is an excerpt from today's eWeek article supporting this position:

To achieve maximum XML performance, bolstered indexing attributes in the technology will enable advanced search functions and a higher degree of filtering. IBM is also adding support for XPath and XQuery data models. This will allow users to create views that involve SQL and XQuery by sending the protocol through DB2's query optimizer for a unified query plan.

Read on..

Virtuoso has been doing this since 2000; unfortunately a lot of

I have always believed that self-annotation will ultimately drive the realization of the semantic web vision. GuruNet is an interesting effort that should lead down this path.

Here is GuruNet's answer to the question: What Is SQL?

The Web Services, XML, and RDF angles should be pretty obvious (I hope!).

BTW - GuruNet does have a sync latency issue re. Wikipedia that it will need to address sooner rather than later.

A quick head to blog dispatch of these thoughts (while they remain fresh):

Data is an expression of feedback; a statement (rightly or wrongly so) about an observation. If you think about it, didn't we used to capture observed data on paper in tabular form (row and columns which are analogous to Relational Database Tables and Columns)?

Information is data in context, or as I would prefer to say: contextualized data. Thus, information provides an understanding of data (provides insight about statements of observation). I also recall a myriad of context oriented hierarchical presentation forms: taxonomies and ontologies or conceptual schemas (nowadays expressed in an hierarchical tree form called XML and persisted for future reference in an XML aware database).

Knowledge isn't contextualized information, and it is certainly distinct from information (contrary to many dictionary definitions as highlighted in this post by Amy Gahran). I prefer to define knowledge as the basis of what you can, will, would, should, or might do with information. And all cases we express our levels knowledge by the way we act on the information (or lack there of) at our disposal. Think about brainstorming for a moment; you are trying to determine a path of action based on information at your disposal, a typical action would be to draw conceptual or topic relationship maps (graphing, with direction driven by the information processing action) on a whiteboard or piece of paper. Expressing, sharing, processing, and persisting these concepts and topics graphs are what the 'Graph Model' based semantic/knowledge database is all about.

Our industry has derived appropriate technology solution realms for Data, Information, and Knowledge Management (although we mix them up more often than not). Thus, there is room for Network, Hierarchical, SQL, XML (Semi-Structured Model), Object, Object-Relational, and Associative Model (graph based modeling of: source, verb, target; analogous to subject, predicate, object as per RDF).

We are spawning data, databases, infobases, knowledgebases, networks, and eventually agents, that will reflect the timeless relationships that exist across; data, information, and knowledge.

..So, using relational storage is inadequate for one reason or another, and IBM has concluded that another approach is necessary. The company’s next generation database will therefore have two storage engines: one relational store and one native XML store. And let me be quite clear about this: these engines will be completely separate, with separate tablespaces, separate indexes (Btrees and so forth on the one hand, and hierarchical on the other), and so on...

Hold on here! IBM only

Stickiness is a defining characteristic of Web 1.0 . It's all about eyeballs (site visitors) which implied ultimately that all early Web business models ended up down the advertising route.

I always felt that Web 1.0 was akin to having a crowd of people at your reception area seeking a look at your corporate brochures, and then someone realizes that you could start selling AD space in these brochures in response to the growing crowd size and frequency of congregation. The long-term folly of this approach is now obvious, as many organizations forgot their core value propositions (expressed via product offerings) in the process and wandered blindly down the AD model cul-de-sac, and we all know what happened down there..

Web 2.0 is taking shape (the inflection is in its latter stages), and the defining characteristics of Web 2.0 are:

1. Fabric of Executable Endpoints
   
2. Semantic Content (the RSS/RDF/Atom/FOAF semantic crumbs emerging from the Blogosphere are great examples of things to come re. XQuery queries over HTTP for instance) Migration from the Web Site (defined by static or dynamic HTML page generation) concept, to that of a "Web Point of Presence" (I don't know if this term will catch on, but the conceptual essence here is factual) that enables an organization to achieve the following:
   
• Package/catalog value proposition (product and services) using RSS/RDF/Atom
  
• Provide SOAP compliant Executable Endpoints (Web Services) for consuming value proposition (as opposed to being distracted by the AD model)
  
• Provide Web Services for consummating contracts associated with core value proposition Identification of internal efficiencies, new products/services that leverage Semantic Content and Web Services, and tangibly exploit:
  
• Composite Web Services construction from legacy monolithic application pools
  
• Standards based (e.g. BPEL) orchestration and integration of disparate composite services (across the Fabric referred to above)

When you factor in all of the above, the real question is whether Google and others are equipped to exploit Web 2.0?  To some degree, is the best answer at the current time as they have commenced the transition from "content only" web site to web platform (via the many Web Services initiatives that expose SOAP and REST interfaces to various services), but there is much more to this journey, and that's the devil in the "competitive landscape details".

From my obviously biased perspective, I think Virtuoso and Yukon+WinFS provide the server models for driving Web 2.0 points of presence (single server instances that implement multiple protocols). Thus, if Google, Yahoo! et al. aren't exploiting these or similar products, then they will be vulnerable over the long term to the competitve challenges that a Web 2.0 landscape will present.

The site has an RSS Feed (but not RSS auto-discovery), its data is available in Excel format (why not XML? This is really a "Save As" issue these days from Excel).

Great site! But it could even be better if XML was used as the data format as opposed to Excel. It would then become a major data source for a myriad of innovative XML data consumption and repurposing demos etc.

Since last fall, I've been recommending Bloglines to first-timers as the fastest and easiest introduction to the subscription side of the blogosphere. Remarkably, this same application also meets the needs of some of the most advanced users. I've now added myself to that list. Hats off to Mark Fletcher for putting all the pieces together in such a masterful way.

What goes around comes around. Five years ago, centralized feed aggregators -- my.netscape.com and my.userland.com -- were the only game in town. Fat-client feedreaders only arrived on the scene later. Because of the well-known rich-versus-reach tradeoffs, I never really settled in with one of those. Most of the time I've used the Radio UserLand reader. It is browser-based, and it normally points to localhost, but I've been parking Radio UserLand on a secure server so that I can read the feeds it aggregates for me from anywhere.

Bloglines takes that idea and runs with it. Like the Radio UserLand reader, it supports the all-important (to me) consolidated view of new items. But its two-pane interface also shows me the list of feeds, highlighting those with new entries, so you can switch between a linear of scan of all new items and random access to particular feeds. Once you've read an item it vanishes, but you can recall already-read items like so:

If a month's worth of some blog's entries produces too much stuff to easily scan, you can switch that blog to a titles-only view. The titles expand to reveal all the content transmitted in the feed for that item.

I haven't gotten around to organizing my feeds into folders, the way other users of Bloglines do, but I've poked around enough to see that Bloglines, like Zope, handles foldering about as well as you can in a Web UI -- which is to say, well enough. With an intelligent local cache it could be really good; more on that later.

Bloglines does two kinds of data mining that are especially noteworthy. First, it counts and reports the number of Bloglines users subscribed to each blog. In the case of Jonathan Schwartz's weblog, for example, there are (as of this moment) 253 subscribers.

Second, Bloglines is currently managing references to items more effectively than the competition. I was curious, for example, to gauge the reaction to the latest salvo in Schwartz's ongoing campaign to turn up the heat on Red Hat. Bloglines reports 10 References. In this case, the comparable query on Feedster yields a comparable result, but on the whole I'm finding Bloglines' assembly of conversations to be more reliable than Feedster's (which, however, is still marked as 'beta'). Meanwhile Technorati, though it casts a much wider net than either, is currently struggling with conversation assembly.

I love how Bloglines weaves everything together to create a dense web of information. For example, the list of subscribers to the Schwartz blog includes: judell - subscribed since July 23, 2004. Click that link and you'll see my Bloglines subscriptions. Which you can export and then -- if you'd like to see the world through my filter -- turn around and import.

Moving my 265 subscriptions into Bloglines wasn't a complete no-brainer. I imported my Radio UserLand-generated OPML file without any trouble, but catching up on unread items -- that is, marking all of each feed's sometimes lengthy history of items as having been read -- was painful. In theory you can do that by clicking once on the top-level folder containing all the feeds, which generates the consolidated view of unread items. In practice, that kept timing out. I finally had to touch a number of the larger feeds, one after another, in order to get everything caught up. A Catch Up All Feeds feature would solve this problem.

Another feature I'd love to see is Move To Next Unread Item -- wired to a link in the HTML UI, or to a keystroke, or ideally both.

Finally, I'd love it if Bloglines cached everything in a local database, not only for offline reading but also to make the UI more responsive and to accelerate queries that reach back into the archive.

Like Gmail, Bloglines is the kind of Web application that surprises you with what it can do, and makes you crave more. Some argue that to satisfy that craving, you'll need to abandon the browser and switch to RIA (rich Internet application) technology -- Flash, Java, Avalon (someday), whatever. Others are concluding that perhaps the 80/20 solution that the browser is today can become a 90/10 or 95/5 solution tomorrow with some incremental changes.

Dare Obasanjo wondered, over the weekend, "What is Google building?" He wrote:

In the past couple of months Google has hired four people who used to work on Internet Explorer in various capacities [especially its XML support] who then moved to BEA; David Bau, Rod Chavez, Gary Burd and most recently Adam Bosworth. A number of my coworkers used to work with these guys since our team, the Microsoft XML team, was once part of the Internet Explorer team. It's been interesting chatting in the hallways with folks contemplating what Google would want to build that requires folks with a background in building XML data access technologies both on the client side, Internet Explorer and on the server, BEA's WebLogic. [Dare Obasanjo]

ListGarden 1.0 released I've just posted the 1.0 version of my ListGarden™ RSS Generator Program. The source code has been released under the GNU GPL license, and it is available for Microsoft Windows, Mac OS X, generic Perl, and server-CGI use. A new feature has been added since the beta release: In addition to creating the XML RSS file, it can also produce an HTML file with the same information as the XML. I discuss RSS feeds in general in this weblog post, as well as the issue of private RSS feeds.

I have little to add to this matter as our understanding and vision is aptly expressed via the architecture and feature set of Virtuoso (this area was actually addressed circa 1999).

We are heading into a era of multi-model databases, these are single database engines that are capable of effectively serving the requirements of the Hierarchical, Network, Relational, and Object database models . As we get closer to the unravelling of universal storage, hopefully this will get clearer.

Back to Dare's commentary:

C.J. Date, one of the most influential names in the relational database world, had some harsh words about XML's encroachment into the world of relational databases in a recent article entitled Date defends relational model  that appeared on SearchDatabases.com. Key parts of the article are excerpted below

Date reserved his harshest criticism for the competition, namely object-oriented and XML-based DBMSs. Calling them "the latest fashions in the computer world," Date said he rejects the argument that relational DBMSs are yesterday's news. Fans of object-oriented database systems "see flaws in the relational model because they don't fully understand it," he said.

Date also said that XML enthusiasts have gone overboard.

"XML was invented to solve the problem of data interchange, but having solved that, they now want to take over the world," he said. "With XML, it's like we forget what we are supposed to be doing, and focus instead on how to do it."

Craig S. Mullins, the director of technology planning at BMC Software and a SearchDatabase.com expert, shares Date's opinion of XML. It can be worthwhile, Mullins said, as long as XML is only used as a method of taking data and putting it into a DBMS. But Mullins cautioned that XML data that is stored in relational DBMSs as whole documents will be useless if the data needs to be queried, and he stressed Date's point that XML is not a real data model.

Craig Mullins points are more straightforward to answer since his comments don't jibe with the current state of the art in the XML world. He states that you can't query XML documents stored in databases but this is untrue. Almost three years ago, I was writing articles about querying XML documents stored in relational databases. Storing XML in a relational database doesn't mean it has to be stored in as an opaque binary BLOB or as a big, bunch of text which cannot effectively be queried. The next version of SQL Server will have extensive capabilities for querying XML data in relational database and doing joins across relational and XML data, a lot of this functionality is described in the article on XML Support in SQL Server 2005. As for XML not having a data model, I beg to differ. There is a data model for XML that many applications and people adhere to, often without realizing that they are doing so. This data model is the XPath 1.0 data model, which is being updated to handled typed data as the XQuery and XPath 2.0 data model.

Now to tackle the meat of C.J. Date's criticisms which is that XML solves the problem of data interchange but now is showing up in the database. The thing first point I'd like point out is that there are two broad usage patterns of XML, it  is used to represent both rigidly structured tabular data (e.g., relational data or serialized objects) and semi-structured data (e.g., office documents). The latter type of data will only grow now that office productivity software like Microsoft Office have enabled users to save their documents as XML instead of proprietary binary formats. In many cases, these documents cannot simply shredded into relational tables. Sure you can shred an Excel spreadsheet written in spreadsheetML into relational tables but is the same really feasible for a Word document written in WordprocessingML? Many enterprises would rather have their important business data being stored and queried from a unified location instead of the current situation where some data is in document management systems, some hangs around as random files in people's folders while some sits in a database management system.

As for stating that critics of the relational model don't understand it, I disagree. One of the major benefits of using XML in relational databases is that it is a lot easier to deal with fluid schemas or data with sparse entries with XML. When the shape of the data tends to change or is not fixed the relational model is simply not designed to deal with this. Constantly changing your database schema is simply not feasible and there is no easy way to provide the extensibility of XML where one can say "after the X element, any element from any namespace can appear". How would one describe the capacity to store “any data” in a traditional relational database without resorting to an opaque blob?

I do tend to agree that some people are going overboard and trying to model their data hierarchically instead of relationally which experience has thought us is a bad idea. Recently on the XML-DEV mailing list entitled Designing XML to Support Information Evolution where Roger L. Costello described his travails trying to model his data which was being transferred as XML in a hierarchical manner. Micheal Champion accurately described the process Roger Costello went through as having "rediscovered the relational model". In a response to that thread I wrote "Hierarchical databases failed for a reason".

Using hierarchy as a primary way to model data is bad for at least the following reasons

1. Hierarchies tend to encourage redundancy. Imagine I have a <Customer> element who has one or more <ShippingAddress> elements as children as well as one or more <Order> elements as children as well. Each order was shipped to an address, so if modelled hierarchically each <Order> element also will have a <ShippingAddress> element which leads to a lot of unnecessary duplication of data.
2. In the real world, there are often multiple groups to which a piece of data belongs which often cannot be modelled with a single hierarchy.  
3. Data is too tightly coupled. If I delete a <Customer> element, this means I've automatically deleted his entire order history since all the <Order> elements are children of <Customer>. Similarly if I query for a <Customer>, I end up getting all the <Order> information as well.

To put it simply, experience has taught the software world that the relational model is a better way to model data than the hierarchical model. Unfortunately, in the rush to embrace XML many a repreating the mistakes from decades ago in the new millenium.

By David Mertz, IBM developerWorks

In Part 2 of a serial article on GUIs and XML configuration data, David discusses how XML is used in the configuration of GUI interfaces. He looks at Mozilla's XML-based User Interface Language (XUL) which allows you to write applications that run without any particular dependency on the choice of underlying operating system. This may seem strange at first, but you'll soon see that this Mozilla project offers powerful tools for GUI building that allow you to develop for an extensive base of installed users. Mozilla is now much more than a browser: it is a whole component and GUI architecture. Indeed, Mozilla is more cross-platform and more widely installed on user systems than probably any other GUI library you are likely to consider. What you might think of as general purpose GUI/widget libraries -- Qt, wxWindows, GTK, FOX, MFC, .NET, Carbon, and so on -- have various advantages and disadvantages. But none of them can be assumed to be already installed across user systems. Many of them are only available on a subset of the platforms Mozilla supports, and most are relatively difficult to install or have licensing issues. Mozilla is worth installing just because it is such a great browser; once you have it, you have a free platform for custom applications. To be completely cross-platform in your Mozilla/XUL applications, you need to restrict yourself to configuring GUIs in XUL and programming their logic in JavaScript.

http://www-106.ibm.com/developerworks/library/x-matters35/

This issue really brings WebDAV into scope as this is the protocol that enables this capability (as covered by Jon's piece), and it is one of the many client and server side protocols implemented by OpenLink Virtuoso (the key to how Virtuoso delivers URI based SQL-XML, XQuery, XPath services).

When you install Virtuoso you simply have to start the Virtuoso server instance to the get WebDAV functionality going. All of Virtuoso's services are advertised at ports, and in the case of WebDAV you will find this at port 8890 if you start the demo database.

To exploit the Virtuoso/WebDAV server from any WebDAV client (or point urls at WebDAV hosted resources) simply do the following:

1. Install Virtuoso and depending on your OS do the following:
• Windows - create a Web Folder that points to a WebDAV server
• Mac OS X - mount a WebDAV folder
• Linux - mount a WebDAV directory (also see the Davfs2 Open Source project)
• You can also make WebDAV client calls from Virtuoso's Stored Procedure Language (Virtuoso PL) or use WebDAV implementations in any development environment of your choice (.NET, Java, .
  
2. Place content that you want to reference in your mails in your WebDAV repository via any of the client side mechanisms described in step 1. You can see the results of this in my earlier blog post, even better pass the url on in an email! Or browse the WebDAV folder (there are some nuggets deliberately left in place :-) )
   
• You could simply save an Office Doc (powerpoint, excel, word etc) to this location and the circulate urls in your mails (this has been standard practice at OpenLink for many years; we even have a full blown portal server that would soon be available as a public service to sharing anything via DAV and as usual some more... stay tuned)
  
3. That's it for any platform (Windows, Linux, Mac OS X, FreeBSD, Solaris, AIX, HP-UX etc.) once you install Virtuoso!

BTW - This blog is WebDAV based (it's a live instance of Virtuoso doing many things; WebDAV, HTTP, SQL-XML based feed generation for ATOM, RSS, Blog Post APIs support (Moveable Type, Metaweblog, Blogger, ATOM), Free Text, XPath, XQuery, and more). 

David Mertz, IBM developerWorks

Over time, XML has permeated many niches. One area where XML is used increasingly is in the configuration of graphical user interfaces, especially in elements that are persistent but should not be fixed at compile-time. In this installment, the author looks at the use of XML in Mac OS X's Aqua GUI, and in the K Desktop Environment (KDE) which is either standard or available in most modern Linux distributions. So far, the use of XML in configuring modern GUIs is a bit haphazard. Most interfaces use XML in some places, but other mechanisms elsewhere. But it's clear that the general movement is toward XML. Clearly, Mac OS X Aqua and KDE each have their own XML philosophy. Mac OS X uses solely XML elements that correspond to broad data types, while everything that is really application-specific or GUI-specific is shunted off into the PCDATA content of container elements. In contrast, KDE's XML feels very usage-specific.

http://www-106.ibm.com/developerworks/xml/library/x-matters34.html

By Nigel McFarlane, devX.com

Microsoft's XAML markup language floats on top of the .NET platform. It is compiled into .NET classes, usually C#, which reduces it to a set of objects. Along with a host of other XML dialects it is an example of a new type of specification for GUIs.

This article takes a look at XAML's tags to see what (if anything) is new in them. There are many such GUI specifications now, a few being Mozilla's XUL, Oracle's UIX, Macromedia's Flex and the XML files created by the Gnome Glade tool. Although not W3C standards, some of these new GUI specifications are already on the W3C standards track. An example is the box model used within Mozilla's XUL, which is headed toward inclusion in future CSS drafts.

The original and most popular source of XML definitions is, however, still the World Wide Web Consortium. The W3C is responsible for formalizing XML and many XML applications such as XHTML and SVG. Given that these standards are mostly complete, do we really need all these new XML GUI dialects?

Microsoft's XAML is a new spin on XML-based GUI description languages, borrowing very little syntax from established standards. Let's see if it's a radical improvement in some way, or if it's merely familiar old friends dressed up in new clothes.

http://www.devx.com/webdev/Article/20834

Their manifesto as presented on their web site:

"OSVB is an independent and open source database created by and for the community. Our goal is to provide accurate, detailed, current, and unbiased technical information".

They also have an XML-RPC based service for programmatic interaction at: http://www.osvdb.org/xmlrpc-server-client-documentation.php

XML based generation of Rich and Native UI's is gathering momentum, it might also be a point to understand the complimentary relationship that exists between XForms and these XML based GUI generators.

BTW - Here is a great XForms presentation that helps aids in the contextualization of my prior comments.

The actual Macromedia MXML (Flex) review by Jon Udell follows:

After a decade of web-style development, I'm sold on the idea of using markup languages to describe the layouts of user interfaces and to coordinate the event-driven code that interconnects widgets and binds them to data. The original expression of that model was HTML and JavaScript, but variations have flourished. Mozilla-based applications have been using XUL (XML User Interface Language) for years. The Laszlo Presentation Server uses a description language called LZX. Microsoft has previewed XAML (Extensible Application Markup Language) for Longhorn.

Now comes MXML (Macromedia Flex Markup Language), the latest development in Macromedia's ongoing quest to reposition the near-ubiquitous Flash player as a general-purpose presentation engine for rich Internet applications. With XML markup at its core, Flex is inherently IDE- friendly, and Macromedia has two IDE initiatives underway. One, code-named Brady, builds on Dreamweaver MX. The other, code-named Partridge, leverages Eclipse.

Full Review: http://www.infoworld.com/article/04/03/29/13TCflex_1.html

Also see XML for UI Languages: http://xml.coverpages.org/userInterfaceXML.html

Nothing stops any of the engines mentioned above (proprietary user interfaces as per the diagram below)

By Lars Marius Garshol, Ontopia Technical Report

Information Architecture is the discipline dealing with the modern version of this problem: how to organize web sites so that users actually can find what they are looking for. Information architects have so far applied known and well-tried tools from library science to solve this problem, and now topic maps are sailing up as another potential tool for information architects. This raises the question of how topic maps compare with the traditional solutions. The paper argues that topic maps go beyond the traditional solutions in the sense that it provides a framework within which they can be represented as they are, but also extended in ways which significantly improve information retrieval. The paper tries to show that topic maps provide a common reference model that can be used to explain how to understand many common techniques from library science and information architecture.

http://www.ontopia.net/topicmaps/materials/tm-vs-thesauri.html

See also (XML) Topic Maps: http://xml.coverpages.org/topicMaps.html

By Jack Vaughan, Application Development Trends

XML has clear benefits as a lingua franca for integration, but it must co-exist with a well-established body of relational DB know-how. More than a few architects are concerned that they will soon have to throw a slew of finely tuned relational queries out the window and start over. JNetDirect recently unveiled software to address RDB-to-XML data mapping. JSQLMapper is a bidirectional data-mapping tool that cuts requirements for custom coding to bring relational data into XML format. With JSQLMapper, developers can create data mappings from existing relational data stores to XML documents.

http://www.adtmag.com/article.asp?id=9092

See also XML and Databases: http://xml.coverpages.org/xmlAndDatabases.html

Why on earth does XML's emergence somehow imply dumping existing investement in relational database queries?

XML provides an alternative, uniform, and widely adopted mechanism for exploiting existing relational data. The emergence of the recent SQL/XML (SQLX) standard should at least bring this into clearer perspective.

Obviously the lack of practical SQLX tutorial material may be the source of some of the confusion about this subject matter, so here are a few urls:

Generating RSS from SQL Data - http://www.openlinksw.com/articles/rssvirtsqlx.htm (this is one of many ways that this can be achieved using OpenLink Virtuoso).

SQLX Tutorials and online demos -
http://demo.openlinksw.com:8890/tutorial/xml/index.vsp?f=1

Databases get a grip on XML
From Inforworld.

The next iteration of the SQL standard was supposed to arrive in 2003. But SQL standardization has always been a glacially slow process, so nobody should be surprised that SQL:2003 ? now known as SQL:200n ? isn?t ready yet. Even so, 2003 was a year in which XML-oriented data management, one of the areas addressed by the forthcoming standard, showed up on more and more developers? radar screens.  >> READ MORE

This article rounds up product for 2003 in the critical area of Enterprise Database Technology. It's certainly provides an apt reflection of how Virtuoso compares with offerings from some the larger (but certainly slower to implement) database vendors in this space. As usual Jon Udell's quote pretty much sums this up:

"While the spotlight shone on the heavyweight contenders, a couple of agile innovators made noteworthy advances in 2003. OpenLink Software?s Virtuoso 3.0, which we reviewed in March, stole thunder from all three major players. Like Oracle, it offers a WebDAV-accessible XML repository. Like DB2 Information Integrator, it functions as database middleware that can perform federated ?joins? across SQL and XML sources. And like the forthcoming Yukon, it embeds the .Net CLR (Common Language Runtime), or in the case of Linux, Novell/Ximian?s Mono."

Albeit still somewhat unknown to the broader industry we have remained true our "innovator" discipline, which still remains our chosen path to market leadership. Thus, its worth a quick Virtuoso release history, and features recap as we get set to up the ante even further in 2004:

1998 - Virtuoso's initial public beta release with functional emphasis on Virtual Database Engine for ODBC and JDBC Data Sources.

1999 - Virtuoso's official commercial release, with emphasis still on Virtual Database functionality for ODBC, JDBC accessible SQL Databases.

2000 - Virtuoso 2.0 adds XML Storage, XPath, XML Schema, XQuery, XSL-T, WebDAV, SOAP, UDDI, HTTP, Replication, Free Text Indexing (*feature update*), POP3, and NNTP support.

2002 - Virtuoso 2.7 extends Virtualization prowess beyond data access via enhancements to its Web Services protocol stack implementation by enabling SQL Stored Procedures to be published as Web Services. It also debuts its Object-Relational engine enhancements that include the incorporation of Java and Microsoft .NET Objects into its User Defined Type, User Defined Functions, and Stored Procedure offerings.

2003 - Virtuoso 3.0 extends data and application logic virtualization into the Application Server realm (basically a Virtual Application server too!), by adding support for ASP.NET, PHP, Java Server Pages runtime hosting (making applications built using any of these languages deployable using Virtuoso across all supported platforms).

Collectively each of these releases have contributed to a very premeditated architecture and vision that will ultimately unveil the inherent power of critical I.S infrastructure virtualization along the following lines; data storage, data access , and application logic via coherent integration of SQL, XML, Web Services, and Persistent Stored Modules (.NET, Java, and other object based component building blocks).

 

XForms Freebie First Eric van der Vlist makes his RELAX NG book freely available, and now Micah Dubinko has done the same re XForms.

RELAX NG is a book in progress written by Eric van der Vlist for O'Reilly and submitted to an open review process. The result of this work will be freely available on the World Wide Web under a Free Documentation Licence (FDL).

The subject of this book, RELAX NG (http://relaxng.org), is a XML schema language developped by the OASIS RELAX NG Technical Committee and recently accepted as Draft International Standard 19757-2 by the Document Description and Processing Languages subcommittee (DSDL) of the ISO/IEC Joint Technical Committee 1 (ISO/IEC JTC 1/SC 34/WG 1).

[via Lost Boy]

It starts here with one of many excerpts from Scoble's blog:

Dave Winer, Jon Udell, and now Gerald Bauer says that Microsoft is killing the Web. Or trying to.

The guys above are pretty seasoned individuals (they save me a lot of writing too amongst other things).

Now here is a response from Microsoft?s Blog evangelist supremo Scoble to their comments and genuine concerns.

OK, let's assume that's true.

Microsoft has 55,000 employees. $50 billion or so in the bank.

Yet what has gotten me to use the Web less and less lately? RSS 2.0.

Seriously. I rarely use the browser anymore (except to post my weblog since I use Radio UserLand).

See the irony there? Dave Winer (who at minimum popularized RSS 2.0) has done more to get me to move away from the Web than a huge international corporation that's supposedly focused on killing the Web.

Now, let's look at what's really going on here. We're going back to being a great platform company. We're trying to provide a platform that lets developers build new applications that are impossible to build on other platforms. At the PDC you saw some of that. New kinds of forms. New kinds of games. New kinds of business apps. New kinds of experiences.

But, we also are looking for ways to make the Web better too. Now, we haven't talked about what we're doing with the browser. I hear that'll come later. Astute Longhorn testers have already seen that we snuck a pop-up ad blocker into the browser without telling anyone about it. Whoa. That means we're gonna turn off MSN's capabilities of selling popup ads.

I hear there's more coming too. But, why should we do it all? Wasn't the point of the past four years to get Microsoft to stop trying to do it all? The DOJ and now the European Union are still after us cause we tried to do it all. Instead, let's just go back and be a great platform company.

We just gave you a great foundation for a killer new kind of application. One that goes FAR beyond HTML. And, even if you stick with Mozilla, your experiences on Longhorn will get better. For instance, fonts are being rendered in the GPU now on Longhorn. Your Web pages will look better and behave better on Longhorn than they will on any other platform. Period.

And wait until Mozilla's and other developers start exploiting things like WinFS to give you new features that display Internet-based information in whole new ways.

If Microsoft really wants to create a better platform shouldn?t this be truly futuristic? If so, then it should issue the first major salvo by dropping the restrictions on Rotor?

We are moving into the distributed component based computing age where runtime environments (.NET CLR, Mono, J2EE, and others) act a Component Execution Junction boxes (instead of the Monolithic Operating Systems of today) in a continuum of services orchestrated by messages in response to events emanating from value consumption requests (what we call application behviour today) from a myriad of value consumers (application users).

There is no need for covert and protracted protection of an obsolete Windows Operating System (the underlying fear that keeps Rotor shackled in my opinion), since its obsolescence is in full motion as Longhorn clearly demonstrates.

Imagine a fusion of sorts across Microsoft .NET, Mono, and Rotor, with a single portable runtime as the end product (slotting nicely into its place in the imminent distributed component and services era). All the benefits of programming language independence in true glory - the ECMA-CLI is all about programming language independence. Now that would be unequivocally revolutionary, and Microsoft would actually be doing what I think it has been desperately trying to achieve for a long time; the delivery of really cool technology that seriously impact us all in a positive way without the usual World Domination Concerns. 

Anyway, back to the current reality where we have covert attempts to lock us all into Windows getting more and more transparent per technology release cycle. The very antithesis of what I espoused in the last paragraph (or dream). I believe that Scoble's instincts lie in this realm too, and you never know this evangelist may turn Messiah :-)

Here's the final excerpt from Scoble?s post:

There's a whole lot of more useful stuff coming. Both for the Web and for newer Internet-centric rich-client approaches. Personally, it's about time. I'm already using the Web less and less thanks to things like RSS 2.0.

I'm watching 636 sites every day. Try to do THAT in your Web browser.

So, yes, blame it on me. I'm trying to kill the Web. Isn't it time to move on? Didn't we move on from the Apple II? Didn't we move on from DOS? Didn't we move on from Windows 3.11? Can't you see a day when we move on from the Web and get something even more fantastic? I can. Dave Winer can. Why not you? [via The Scobleizer Weblog]

If you kill the Web en route to getting us a Portable Execution Junction box from Microsoft, I think you would have served mankind pretty damned well. We won't have to gripe about Web 1.0 (Browser Driven Web) because we would be well into Web 2.0 and beyond (which doesn?t define the Web experience predominantly via browsing).

NETWORK WORLD NEWSLETTER: MARK GIBBS ON WEB APPLICATIONS

Today's focus: A Virtuoso of a server

By Mark Gibbs

One of the bigger drags of Web applications development is that building a system of even modest complexity is a lot like herding cats - you need a database, an applications server, an XML engine, etc., etc. And as they all come from different vendors you are faced with solving the constellation of integration issues that inevitably arise.

If you are lucky, your integration results in a smoothly functioning system. If not, you have a lot of spare parts flying in loose formation with the risk of a crash and burn at any moment.

An alternative is to look for all of these features and services in a single package but you'll find few choices in this arena.

One that is available and looks very promising is OpenLink's Virtuoso (see links below).

Virtuoso is described as a cross platform (runs on Windows, all Unix flavors, Linux, and Mac OS X) universal server that provides databases, XML services, a Web application server and supporting services all in a single package.

OpenLink's list of supported standards is impressive and includes .Net, Mono, J2EE, XML Web Services (Simple Object Application Protocol, Web Services Description Language, WS-Security, Universal Description, Discovery and Integration), XML, XPath, XQuery, XSL-T, WebDav, HTTP, SMTP, LDAP, POP3, SQL-92, ODBC, JDBC and OLE-DB.

Virtuoso provides an HTTP-compliant Web Server; native XML document creation, storage and management; a Web services platform for creation, hosting and consumption of Web services; content replication and synchronization services; free text index server, mail delivery and storage and an NNTP server.

Another interesting feature is that with Virtuoso you can create Web services from existing SQL Stored Procedures, Java classes,

C++ classes, and 'C' functions as well as create dynamic XML

documents from ODBC and JDBC data sources.

This is an enormous product and implies a serious commitment on the part of adopters due to its scope and range of services.

Virtuoso is enormous by virtue of its architectural ambitions, but actual disk requirements are

October the 1st is an emotional day for many Nigerians, especially those of us in the Diaspora. Our country remains a paradox as the excerpts below attest:

The more popular view of Nigerians as a result of the proliferation of 419 scams (the mangled by-product of misdirected intellectual prowess and the boundless depths of greed -- which applies to perpetrators and victims alike).

The Nigerian SCO Connection "I AM MR. DARL MCBRIDE CURRENTLY SERVING AS THE PRESIDENT AND CHIEF EXECUTIVE OFFICER OF THE SCO GROUP ..." [via Be Blogging]

Funny! But many a truth is told in jest (I think that's how the quote goes); this one is pretty damned poignant.

Unbeknownst to many, there are other views of Nigeria (unfortunately these aren't the norm).

The call for optimism by our president (he doesn't support or condone the 419 nonsense):

 

President Olusegun Obasanjo urged Nigerians to change their ways and be optimistic about the future as Nigeria marks its 43rd Independence anniversary.  Read on

[via Odili.net � this site desperately needs RSS!]

 

There is an increasing pool of key high-tech players of Nigerian decent (and nationality) making constructive impact on the high-tech industry (making it less lonely for myself and other Nigerians in the high-tech arena):

 

Dare Obasanjo is a member of Microsoft's WebData team, which among other things develops the components within the System.Xml and System.Data namespace of the .NET Framework, Microsoft XML Core Services (MSXML), and Microsoft Data Access Components (MDAC). More of Dare's writings on XML can be found on his Extreme XML column on MSDN.

 

Uche Ogbuji is a consultant and co-founder of Fourthought Inc., a consulting firm specializing in XML solutions for enterprise knowledge management applications. Fourthought develops 4Suite, the open source platform for XML middleware. Mr. Ogbuji is a Computer Engineer and writer born in Nigeria, living and working in Boulder, Colorado, USA.
Website: http://www.fourthought.com/