Your Life, Profession, Web, and Internet do not need to become mutually exclusive due to "information overload".

Solution:

A platform or service that delivers a point of online presence that embodies the fundamental separation of: Identity, Data Access, Data Representation, Data Presentation, by adhering to Web and Internet protocols.

How:

Typical post installation (Local or Cloud) task sequence:

1. Identify myself (happens automatically by way of registration)
2. If in an LDAP environment, import accounts or associate system with LDAP for account lookup and authentication
3. Identify Online Accounts (by fleshing out profile) which also connects system to online accounts and their data
4. Use Profile for granular description (Biography, Interests, WishList, OfferList, etc.)
5. Optionally upstream or downstream data to and from my online accounts
6. Create content Tagging Rules
7. Create rules for associating Tags with formal URIs
8. Create automatic Hyperlinking Rules for reuse when new content is created (e.g. Blog posts)
9. Exploit Data Portability virtues of RSS, Atom, OPML, RDFa, RDF/XML, and other formats for imports and exports
10. Automatically tag imported content
11. Use function-specific helper application UIs for domain specific data generation e.g. AddressBook (optionally use vCard import), Calendar (optionally use iCalendar import), Email, File Storage (use WebDAV mount with copy and paste or HTTP GET), Feed Subscriptions (optionally import RSS/Atom/OPML feeds), Bookmarking (optionally import bookmark.html or XBEL) etc..
12. Optionally enable "Conversation" feature (today: Social Media feature) across the relevant application domains (manage conversations under covers using NNTP, the standard for this functionality realm)
13. Generate HTTP based Entity IDs (URIs) for every piece of data in this burgeoning data space
14. Use REST based APIs to perform CRUD tasks against my data (local and remote) (SPARQL, GData, Ubiquity Commands, Atom Publishing)
15. Use OpenID, OAuth, FOAF+SSL, FOAF+SSL+OpenID for accessing data elsewhere
16. Use OpenID, OAuth, FOAF+SSL, FOAF+SSL+OpenID for Controlling access to my data (Self Signed Certificate Generation, Browser Import of said Certificate & associated Private Key, plus persistence of Certificate to FOAF based profile data space in "one click")
17. Have a simple UI for Entity-Attribute-Value or Subject-Predicate-Object arbitrary data annotations and creation since you can't pre model an "Open World" where the only constant is data flow
18. Have my Personal URI (Web ID) as the single entry point for controlled access to my HTTP accessible data space

I've just outlined a snippet of the capabilities of the OpenLink Data Spaces platform. A platform built using OpenLink Virtuoso, architected to deliver: open, platform independent, multi-model, data access and data management across heterogeneous data sources.

All you need to remember is your URI when seeking to interact with your data space.

Related

1. Get Yourself a URI (Web ID) in 5 Minutes or Less!
2. Various posts over the years about Data Spaces
3. Future of Desktop Post
4. Simplify My Life Post by Bengee Nowack

A pre-installed edition of Virtuoso for Amazon's EC2 Cloud platform.

What does it offer?

1. Low cost entry point to a game-changing Web 3.0+ (and beyond) platform that combines SQL, RDF, XML, and Web Services functionality
2. Flexible variable cost model (courtesy of EC2 DevPay) tightly bound to revenue generated by your services
3. Delivers federated and/or centralized model flexibility for you SaaS based solutions
4. Simple entry point for developing and deploying sophisticated database driven applications (SQL or RDF Linked Data Web oriented)
5. Complete framework for exploiting OpenID, OAuth (including Role enhancements) that simplifies exploitation of these vital Identity and Data Access technologies
6. Easily implement RDF Linked Data based Mail, Blogging, Wikis, Bookmarks, Calendaring, Discussion Forums, Tagging, Social-Networking as Data Space (data containers) features of your application or service offering
7. Instant alleviation of challenges (e.g. service costs and agility) associated with Data Portability and Open Data Access across Web 2.0 data silos
8. LDAP integration for Intranet / Extranet style applications.

From the DBMS engine perspective it provides you with one or more pre-configured instances of Virtuoso that enable immediate exploitation of the following services:

1. RDF Database (a Quad Store with SPARQL & SPARUL Language & Protocol support)
2. SQL Database (with ODBC, JDBC, OLE-DB, ADO.NET, and XMLA driver access)
3. XML Database (XML Schema, XQuery/Xpath, XSLT, Full Text Indexing)
4. Full Text Indexing.

From a Middleware perspective it provides:

1. RDF Views (Wrappers / Semantic Covers) over SQL, XML, and other data sources accessible via SOAP or REST style Web Services
2. Sponger Service for converting non RDF information resources into RDF Linked Data "on the fly" via a large collection of pre-installed RDFizer Cartridges.

From the Web Server Platform perspective it provides an alternative to LAMP stack components such as MySQL and Apace by offering

1. HTTP Web Server
2. WebDAV Server
3. Web Application Server (includes PHP runtime hosting)
4. SOAP or REST style Web Services Deployment
5. RDF Linked Data Deployment
6. SPARQL (SPARQL Query Language) and SPARUL (SPARQL Update Language) endpoints
7. Virtuoso Hosted PHP packages for MediaWiki, Drupal, Wordpress, and phpBB3 (just install the relevant Virtuoso Distro. Package).

From the general System Administrator's perspective it provides:

1. Online Backups (Backup Set dispatched to S3 buckets, FTP, or HTTP/WebDAV server locations)
2. Synchronized Incremental Backups to Backup Set locations
3. Backup Restore from Backup Set location (without exiting to EC2 shell).

Higher level user oriented offerings include:

1. OpenLink Data Explorer front-end for exploring the burgeoning Linked Data Web
2. Ajax based SPARQL Query Builder (iSPARQL) that enables SPARQL Query construction by Example
3. Ajax based SQL Query Builder (QBE) that enables SQL Query construction by Example.

For Web 2.0 / 3.0 users, developers, and entrepreneurs it offers it includes Distributed Collaboration Tools & Social Media realm functionality courtesy of ODS that includes:

1. Point of presence on the Linked Data Web that meshes your Identity and your Data via URIs
2. System generated Social Network Profile & Contact Data via FOAF?
3. System generated SIOC (Semantically Interconnected Online Community) Data Space (that includes a Social Graph) exposing all your Web data in RDF Linked Data form
4. System generated OpenID and automatic integration with FOAF
5. Transparent Data Integration across Facebook, Digg, LinkedIn, FriendFeed, Twitter, and any other Web 2.0 data space equipped with RSS / Atom support and/or REST style Web Services
6. In-built support for SyncML which enables data synchronization with Mobile Phones.

How Do I Get Going with It?

• Standard Installation Guide
• Personal or Service Specific DBpedia Installation Guide

From the RWW Top-Down category, which I interpret as: technologies that produce RDF from non RDF data sources. Our product portfolio is comprised of the following; Virtuoso Universal Server, OpenLink Data Spaces, OpenLink Ajax Toolkit, and OpenLink Data Explorer (which includes ubiquity commands).

Virtuoso Universal Server functionality summary:

1. Generation of RDF Linked Data Views of SQL, XML, and Web Services in general
2. Deployment of RDF Linked Data
3. "On the Fly" generation of RDF Linked Data from Document Web information resources (i.e. distillation of entities from their containers e.g. Web pages) via Cartridges / Drivers
4. SPARQL query language support
5. SPARQL extensions that bring SPARQL closer to SQL e.g Aggregates, Update, Insert, Delete Named Graph support (i.e. use of logical names to partition RDF data within Virtuoso's multi-model dbms engine)
6. Inference Engine (currently in use re. DBpedia via Yago and UMBEL)
7. Host and exposes data from Drupal, Wordpress, MediaWiki, phpBB3 as RDF Linked Data via in-built support for PHP runtime
8. Available as an EC2 AMI
9. etc..

OpenLink Data Spaces functionality summary:

1. Simple mechanism for Linked Data Web enabling yourself by giving you an HTTP based User ID (a de-referencable URI) that is linked to a FOAF based Profile page and OpenID
2. Binds all your data sources (blogs, wikis, bookmarks, photos, calendar items etc. ) to your URI so can "Find" things by only remembering your URI
3. Makes your profile page and personal URI the focal point of Linked Data Web presence
4. Delivers Data Portability (using data access by value or data access by reference) across data silos (e.g. Web 2.0 style social networks)
5. Allows you make annotations about anything in your own Data Space(s) on the Web without exposure to RDF markup
6. A Briefcase feature that provides a WebDAV driven RDF Linked Data variant of functionality seen in Mac OS X Spotlight and WinFS with the addition of SPARQL compliance
7. Automatically generates RDFa in its (X)HTML pages
8. Blog, Wiki, WebDAV File Server, Shared Bookmarks, Calendar, and other applications that look and feel like Web 2.0 counterparts but emitt RDF Linked Data amongst a plethora of data exchange formats
9. Available as an EC2 AMI
10. etc..

OpenLink Ajax Toolkit functionality summary:

1. Provides binding to SQL, RDF, XML, and Web Services via Ajax Database Connectivity Layer (you only need an ODBC, JDBC, OLE-DB, ADO.NET, XMLA Driver, or Web Service on the backend for dynamic data access from Javascript)
2. All controls are Ajax Database Connectivity bound (widgets get their data from Ajax Database Connectivity data sources)
3. Bundled with Virtuoso and ODS installations.
4. etc.

OpenLink Data Explorer functionality summary

1. Distills entities associated with information resource style containers (e.g. Web Pages or files) as RDF Linked Data
2. Exposes the RDF based Linked Data graph associated with information resources (see the Linked Data behind Web pages)
3. Ubiquity commands for invoking the above
4. Available as a Hosted Service or Firefox Extension
5. Bundled with Virtuoso and ODS installations
6. etc.

Note:

Of course you could have simply looked up OpenLink Software's FOAF based Profile page (*note the Linked Data Explorer tab*), or simply passed the FOAF profile page URL to a Linked Data aware client application such as: OpenLink Data Explorer, Zitgist Data Viewer, Marbles, and Tabulator, and obtained information. Remember, OpenLink Software is an Entity of Type: foaf:Organization, on the burgeoning Linked Data Web :-)

Related

• Linked Data Planet Keynote (RDFa based remix edition)
• On The Cusp: A Global Review of the Semantic Web Industry.

Ivan's presentation titled: State of the Semantic Web, is a must view for those who need a quick update on where things are re. the Semantic Web in general.

I also liked the fact that in proper "Lead by example" manner, his presentation isn't PDF or PPT based, it's a Web Document :-)

Hint: as per usual, this post contains a Linked Data demo nugget. This time around, it's in the form of a shared calendar covering a large number of Semantic Web Technology events. All I had to do was subscribe to a number of WebDAV accessible iCal files from my Calendar Data Space and the platform did the rest i.e. produce Linked Data Objects for events associated with a plethora of conferences.

If you assimilate Ivan's presentation properly, you will note I've just generated, and shared, a large number of URIs covering a range of conference events. Thus, you can extend my contributions (thereby enriching the GGG) by simply associating additional data from your Linked Data Space with mine. All you have to do is use my calendar data objects URIs in your statements.

-- Linked Data does not implicitly imply making all your data public

-- Linked Data principles benefit Intranet and Extranet style data integration (trumps alternative distributed database integration approaches any day)

-- Business exploitation of Linked Data on the Web will certainly be driven by the correlation of opportunity costs (which is more than likely what Alex meant by "use cases") associated with the lack of URIs originating from the domain of a given business (Tom Heath: also effectively alluded to this via his BBC and URI land grab anecdotes; same applies Georgi's examples)

-- History is a great tutor, answers to many of today's problems always lie somewhere in plain sight of the past.

Of course, I also believe that Linked Data serves Web Data Integration across the Internet very well too, and the fact that it will be beneficial to businesses in a big way. No individual or organization is an island, I think the Internet and Web have done a good job of demonstrating that thus far :-) We're all data nodes in a Giant Global Graph.

Daniel lewis did shed light on the read-write aspects of the Linked Data Web, which is actually very close to the callout for a Wikipedia for Data. TimBL has been working on this via Tabulator (see Tabulator Editing Screencast), Bengamin Nowack also added similar functionality to ARC, and of course we support the same SPARQL UPDATE into an RDF information resource via the RDF Sink feature of our WebDAV and ODS-Briefcase implementations.

- Produce RDF Linked Data from non RDF Data Sources (Heterogeneous SQL, XML, Web Services)

- Provide highly scalable RDF Data Management via a Quad Store (DBpedia is an example of a live demonstration)

- Sophisticated Deployment of Linked Data that exploits the power of SPARQL

- Powerful WebDAV innovations that simplify read-write mode interaction with Linked Data

- Use Web Data Virtualization to address the pain and frustration associated with Web Data Silos (e.g. OpenLink Data Spaces layer stop Virtuoso that delivers Personal Data Spaces / Unified Storage in the Clouds)

- Deliver a Linked Data development and deployment platform to .NET (VB, C#) , Java, PHP, Ruby, Perl, Python, 'C', C++, and other developers

- More...

Web Server:

- HTTP 1.1 compliant Transparent content-negotiation in URL-rewrite rules for Linked Data Deployment.

RDF Data Management:

- New providers for the Jena, Sesame and Redland frameworks

- support for SPARQL INSERT and UPDATE via HTTP POST

- New SPARQL-BI extenstions that make Business Intelligence feasible via SPARQL

- new "rdf_sink" folder for handling HTTP PUTs into WebDAV that automatically sync with Quad Store.

- There are new Sponger (RDFizer) cartridges that map Amazon book-search results to the Biliographic Ontology, supports production of Linked Data from OAI, XBRL, and Yahoo finance data sources.

- HTTPS protocol support added to Sponger

- performance optimizations for SPARQL `DESCRIBE' and `CONSTRUCT', alongside general performance enhancements for RDF data set loading.

Core DBMS Engine:

- PHP hosting a module re-implemented as a Virtuoso plugin inline with otherlanguage hosting modules

- improved deadlock condtion management

- enhanced POP and FTP server side protocol implementations that allow larger data transfers.

Additional Information

- DBpedia URI

- Product Home Page

- Wikipedia Page

- Virtuoso 5.0.4 Press Release

* 64-bit Integer Support

* RDF Sink Folders for WebDAV - enabling RDF Quad Store population by simply dropping RDF files into WebDAV or via HTTP (meaning you can use CURL as an RDF in put mechanism for instance)

* Additional Sponger Cartridges from Audio binary files (i.e ID3 tag extraction and Music Ontology mapping which exposes the fine details of music as RDF based Structured Data; one for the DJs & Remixers out there!)

* New Sponger Cartridges for Facebook, Freebase, Wikipedia, GRDDL, RDFa, eRDF and more

* Support for PHP 5.2 runtime hosting (Virtuoso is a bona fide deployment platform for: Wordpress, MediaWiki, phpBB, Drupal etc.)

* Enhanced UI for managing RDF Linked Data deployment (covering Multi Homed domains, Virtual Directories associated with URL-rewrite rules

* Demonstration Database includes SQL-RDF Views & SQL Table samples for the THALIA Web Data Integration benchmark and test-suite

* Tutorial Application includes Linked Data style SQL-RDF Views for the Northwind SQL DBMS schema (which is the same as the standard Virtuoso demo atabase schema)

* SQL-RDF Views implementation of the TPC-D benchmark (Yes, we can run this grueling SQL benchmark via RDF views of SQL Data!)

* A new Amazon EC2 Image for Virtuoso that enables you to instantiate a fully configured instance comprising the Virtuoso core, OpenLink Data Spaces platform and the OpenLink Ajax Toolkit (OAT) (we now have bona fide Data Spaces in the Clouds as an addition to the emerging Semantic Data Web mesh).

Download Lnks:

* Open Source Edition

* Commercial Edition

Jon Udell recently penned a post titled: The Fourth Platform. The post arrives at a spookily coincidental time (this happens quite often between Jon and I as demonstrated last year during our podcast; the "Fourth" in his Innovators Podcast series).

The platform that Jon describes is "Cloud Based" and comprised of Storage and Computation. I would like to add Data Access and Management (native and virtual) under the fourth platform banner with the end product called: "Cloud based Data Spaces".

As I write, we are releasing a Virtuoso AMI (Amazon Image) labeled: virtuoso-dataspace-server. This edition of Virtuoso includes the OpenLink Data Spaces Layer and all of the OAT applications we've been developing for a while.

What Benefits Does this offer?

1. Personal Data Spaces in the Cloud - a place where you can control and consolidate data across your Blogs, Wikis, RSS/Atom Feed Subscriptions, Shared Bookmarks, Shared Calendars, Discussion Threads, Photo Galleries etc
2. All the data in your Data Space is SPARQL or GData accessible.
3. All of the data in your Personal Data Space is Linked Data from the get go. Each Item of data is URI addressable
4. SIOC support - your Blogs, Wikis, Bookmarks etc.. are based on the SIOC ontology for Semantically Interlinking Online Communities (think: Open social-graph++)
5. FOAF support - your FOAF Profile page provides a URI that is an in-road to all Data in your Data Space.
6. OpenID support - your Personal Data Space ID is usable wherever OpenID is supported. OpenID and FOAF are integrated as per latest FOAF specs
7. Two Integration with Facebook - You can access your Data Space from Facebook or access Facebook from your Data Space
8. Unified Storage - The WebDAV based filesystem provides Cloud Storage that's integrated with Amazon S3; It also exposes all of your Data Space data via a traditional filesystem UI (think virtual Spotlight); You can also mount this drive to your local filesystem via your native operating system's WebDAV support
9. SyncML - you can sync calendar and contact details with your Data Space in the cloud from your Mobile phone.
10. A practical Semantic Data Web solution - based on Web Infrastructure and doesn't require you to do anything beyond exposing URIs for data in your Data Spaces.

EC2-AMI Details:

AMI ID: ami-e2ca2f8b

Manifest file: virtuoso-images/virtuoso-dataspace-server.manifest.xml

Installation Guide:

1. Get an Amazon Web Services (AWS) account
2. Signup for S3 and EC2 services
3. Install the EC2 plugin for Firefox
4. Start the EC2 plugin
5. Locate the row containing ami-7c31d515  Manifest virtuoso-test/virtuoso-cloud-beta-9-i386.manifest.xml (sort using the AMI ID or Manifest Columns or search on pattern: virtuoso, due to name flux)
6. Start the Virtuoso Data Space Server AMI
7. Wait 4-5 minutes (*take a few minutes to create the pre-configured Linux Image*)
8. Connect to http://http://your-ec2-instance-cname:8890/ Log in with user/password dba/dba
9. Go to the Admin UI (Virtuoso Conductor) and change the PWDs for the 'dba' and 'dav' accounts (*Important!*)
10. Give the "SPARQL" user "SPARQL_UPDATE" privileges (required if you want to exploit the in-built Sponger Middleware)
11. Click on the ODS (OpenLink Data Spaces) link to start an Personal Editon of OpenLink Data Spaces (or go to: http://your-ec2-instance-cname/dataspace/ods/index.html)
12. Log-in using the username and password credentials for the 'dav' account (or register a new user note: OpenID is an option here also) Create an Data Space Application Instance by clicking on a Data Space App. Tab
13. Import data from your existing Web 2.0 style applications into OpenLink Data Spaces e.g. subscribe to a few RSS/Atom feeds via the "Feeds Manager" application or import some Bookmarks using the "Bookmarks" application
14. Then look at the imported data in Linked Data form via your ODS generated URIs based on the patterns: http://your-ec2-instance-cname/dataspace/person/your-ods-id#this (URI for You the Person), http://your-ec2-instance-cname/dataspace/person/your-ods-id (FOAF File URI), http://your-ec2-instance-cname/dataspace/your-ods-id (SIOC File URI)
    

(OAT) from your Data Space instance

1. http://:8890/oatdemo - Entire OAT Demo Collection
2. http://:8890/rdfbrowser - RDF Browser
3. http://:8890/isparql - SPARQL Query Builder (iSPARQL)
4. http://:8890/qbe - SQL Query Builder (iSQL)
5. http://:8890/formdesigner - Forms Builder (for building Meshups based on RDF, SQL, or Web Servives Data Souces)
6. http://:8890/dbdesigner - SQL DB Schema Designer (note a Visual SQL-RDF Mapper is also on it's way
7. http://:8890/DAV/JS/ - To view the OAT Tree (there are some experimental demos that are missing from the main demo app etc..)

There's more to come!

A vital component of the new Virtuoso release is the finalization of our SQL to RDF mapping functionality -- enabling the declarative mapping of SQL Data to RDF. Additional technical insight covering other new features (delivered and pending) is provided by Orri Erling, as part of a series of post-Banff posts.

Why is SQL to RDF Mapping a Big Deal?

A majority of the world's data (especially in the enterprise realm) resides in SQL Databases. In addition, Open Access to the data residing in said databases remains the biggest challenge to enterprises for the following reasons:

1. SQL Data Sources are inherently heterogeneous because they are acquired with business applications that are in many cases inextricably bound to a particular DBMS engine
2. Data is predictably dirty
3. DBMS vendors ultimately hold the data captive and have traditionally resisted data access standards such as ODBC (*trust me they have, just look at the unprecedented bad press associated with ODBC the only truly platform independent data access API. Then look at how this bad press arose..*)

Enterprises have known from the beginning of modern corporate times that data access, discovery, and manipulation capabilities are inextricably linked to the "Real-time Enterprise" nirvana (hence my use of 0.0 before this becomes 3.0).

In my experience, as someone whose operated in the data access and data integration realms since the late '80s, I've painfully observed enterprises pursue, but unsuccessfully attain, full control over enterprise data (the prized asset of any organization) such that data-, information-, knowledge-workers are just a click away from commencing coherent platform and database independent data drill-downs and/or discovery that transcend intranet, internet, and extranet boundaries -- serendipitous interaction with relevant data, without compromise!

Okay, situation analysis done, we move on..

At our most recent (12th June) monthly Semantic Web Gathering, I unveiled to TimBL and a host of other attendees a simple, but powerful, demonstration of how Linked Data, as an aspect of the Semantic Data Web, can be applied to enterprise data integration challenges.

Actual SQL to RDF Mapping Demo / Experiment

Hypothesis

Apparatus

Experiment / Demo

1. Declaratively map the Northwind SQL Schema to RDF using the Virtuoso Meta Schema Language (see: Virtuoso PL based Northwind_SQL_RDF script)
2. Start browsing the data by clicking on the URIs that represent the RDF Data Model Entities resulting from the SQL to RDF Mapping

Observations

1. Via a single Data Link click I was able to obtain specific information about the Customer represented by the URI "ALFKI" (act of URI Dereferencing as you would an Object ID in an Object or Object-Relational Database)
2. Via a Dynamic Data Page I was able to explore all the entity relationships or specific entity data (i.e Exploratory or Entity specific dereferencing) in the Northwind Data Space
3. I was able to perform similar exploration (as per item 2) using our OpenLink Browser.

Conclusions

The vision of data, information, or knowledge at your fingertips is nigh! Thanks to the infrastructure provided by the Semantic Data Web (URIs, RDF Data Model, variety of RDF Serialization Formats[1][2][3], and Shared Data Dictionaries / Schemas / Ontologies [1][2][3][4][5]) it's now possible to Virtualize enterprise data from the Physical Storage Level, through the Logical Data Management Levels (Relational), up to a Concrete Conceptual Model (Graph) without operating system, development environment or framework, or database engine lock-in.

Next Steps

We produce a shared ontology for the CRM and Business Reporting Domains. I hope this experiment clarifies how this is quite achievable by converting XML Schemas to RDF Data Dictionaries (RDF Schemas or Ontologies). Stay tuned :-)

Also watch TimBL amplify and articulate Linked Data value in a recent interview.

Other Related Matters

To deliver a mechanism that facilitates the crystallization of this reality is a contribution of boundless magnitude (as we shall all see in due course). Thus, it is easy to understand why even "her majesty", the queen of England, simply had to get in on the act and appoint TimBL to the "British Order of Merit" :-)

Note: All of the demos above now work with IE & Safari (a "remember what Virtuoso is epiphany") by simply putting Virtuoso's DBMS hosted XSLT engine to use :-) This also applies to my earlier collection of demos from the Hello Data Web and other Data Web & Linked Data related demo style posts.

Note: the enhanced hyperlink (typed data link) lookup presents options to perform an Explore (all data about subject across Domains in the data space i.e. data links to and from Subject), Dereference (specific data in the Subject's Domain i.e. data links originating from subject).

1. Diana Ross
2. Paul McCartney
3. The Beatles
4. Madonna

I built these Linked Data Pages by simply doing the following:

1. Open up our OAT based iSPARQL (Interactive SPARQL Query By Example) Tool
2. Paste a URI of Interest into the Data Source URI input field
3. Execute the Query (hitting the ">" button)
4. Saving the Query to WebDAV as a Linked Data Page (or what I initial called Dynamic Data Web pages in my Hello Data Web series of posts).
5. Share your Data, Information, Knowledge with others via URIs (as shown in the section above).

Now that broader understanding of the Semantic Data Web is emerging, I would like to revisit the issue of "Data Spaces".

A Data Space is a place where Data Resides. It isn't inherently bound to a specific Data Model (Concept Oriented, Relational, Hierarchical etc..). Neither is it implicitly an access point to Data, Information, or Knowledge (the perception is purely determined through the experiences of the user agents interacting with the Data Space.

A Web Data Space is a Web accessible Data Space.

Real world example:

Today we increasing perform one of more of the following tasks as part of our professional and personal interactions on the Web:

1. Blog via many service providers or personally managed weblog platforms
2. Create Event Calendars via Upcoming.com and Eventful
3. Maintain and participate in Social Networks (e.g. Facebook, Orkut, MySpace)
4. Create and Participate in Discussions (note: when you comment on blogs or wikis for instance, you are participating in, or creating, a conversation)
5. Track news by subscribing to RSS 1.0, RSS 2.0, or Atom Feeds
6. Share Bookmarks & Tags via Del.icio.us and other Services
7. Share Photos via Flickr
8. Buy, Review, or Search for books via Amazon
9. Participates in auctions via eBay
10. Search for data via Google (of course!)

John Breslin has nice a animation depicting the creation of Web Data Spaces that drives home the point.

Unfortunately, what isn't as obvious to many netizens, is the fact that each of the activities above results in the creation of data that is put into some context by you the user. Even worse, you eventually realize that the service providers aren't particularly willing, or capable of, giving you unfettered access to your own data. Of course, this isn't always by design as the infrastructure behind the service can make this a nightmare from security and/or load balancing perspectives. Irrespective of cause, we end up creating our own "Data Spaces" all over the Web without a coherent mechanism for accessing and meshing these "Data Spaces".

Data Spaces on the Web that provide granular access to RDF Data.

Short History

In anticipation of this the "Web Data Silo" challenge (an issue that we tackled within internal enterprise networks for years) we commenced the development (circa. 2001) of a distributed collaborative application suite called OpenLink Data Spaces (ODS). The project was never released to the public since the problems associated with the deliberate or inadvertent creation of Web Data silos hadn't really materialized (silos only emerged in concreted form after the emergence of the Blogosphere and Web 2.0). In addition, there wasn't a clear standard Query Language for the RDF based Web Data Model (i.e. the SPARQL Query Language didn't exist).

Today, ODS is delivered as a packaged solution (in Open Source and Commercial flavors) that alleviates the pain associated with Data Space Silos that exist on the Web and/or behind corporate firewalls. In either scenario, ODS simply allows you to create Open and Secure Data Spaces (via it's suite of applications) that expose data via SQL, RDF, XML oriented data access and data management technologies. Of course it also enables you to integrates transparently with existing 3rd party data space generators (Blogs, Wikis, Shared Bookmrks, Discussion etc. services) by supporting industry standards that cover:

1. Content Publishing - Atom, Moveable Type, MetaWeblog, Blogger protocols
2. Content Syndication Formats - RSS 1.0, RSS 2.0, Atom, OPML etc.
3. Data Management - SQL, RDF, XML, Free Text
4. Data Access - SQL, SPARQL, GData, Web Services (SOAP or REST styles), WebDAV/HTTP
5. Semantic Data Web Middleware - GRDDL, XSLT, SPARQL, XPath/XQuery, HTTP (Content Negotiation) for producing RDF from non RDF Data ((X)HTML, Microformats, XML, Web Services Response Data etc).

Thus, by installing ODS on your Desktop, Workgroup, Enterprise, or public Web Server, you end up with a very powerful solution for creating Open Data access oriented presence on the "Semantic Data Web" without incurring any of the typically assumed "RDF Tax".

Naturally, ODS is built atop Virtuoso and of course it exploits Virtuoso's feature-set to the max. It's also beginning to exploit functionality offered by the OpenLink Ajax Toolkit (OAT).

SPARQL (query language for the Semantic Web) basically enables me to query a collection of typed links (predicates/properties/attributes) in my Data Space (ODS based of course) without breaking my existing local bookmarks database or the one I maintain at del.icio.us.

I am also demonstrating how Web 2.0 concepts such as Tagging mesh nicely with the more formal concepts of Topics in the Semantic Web realm. The key to all of this is the ability to generate RDF Data Model Instance Data based on Shared Ontologies such as SIOC (from DERI's SIOC Project) and SKOS (again showing that Ontologies and Folksonomies are complimentary).

This demo also shows that Ajax also works well in the Semantic Web realm (or web dimension of interaction 3.0) especially when you have a toolkit with Data Aware controls (for SQL, RDF, and XML) such as OAT (OpenLink Ajax Toolkit). For instance, we've successfully used this to build a Visual Query Building Tool for SPARQL (alpha) that really takes a lot of the pain out of constructing SPARQL Queries (there is much more to come on this front re. handling of DISTINCT, FILTER, ORDER BY etc..).

For now, take a look at the SPARQL Query dump generated by this SIOC & SKOS SPARQL QBE Canvas Screenshot.

You can cut and paste the queries that follow into the Query Builder or use the screenshot to build your variation of this query sample. Alternatively, you can simply click on *This* SPARQL Protocol URL to see the query results in a basic HTML Table. And one last thing, you can grab the SPARQL Query File saved into my ODS-Briefcase (the WebDAV repository aspect of my Data Space).

Note the following SPARQL Protocol Endpoints:

1. MyOpenLink Data Space
2. Experimental Data Space SPARQL Query Builder (you need to register at http://myopenlink.net:8890/ods to use this version)
3. Live Demo Sever
4. Demo Server SPARQL Query Builder (use: demo for both username and pwd when prompted)

My beautified Version of the SPARQL Generated by QBE (you can cut and paste into "Advanced Query" section of QBE) is presented below:

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX dct: <http://purl.org/dc/elements/1.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>


SELECT distinct 
       ?forum_name, 
       ?owner, 
       ?post, 
       ?title, 
       ?link, 
       ?url, 
       ?tag
FROM <http://myopenlink.net/dataspace>
WHERE {
       ?forum a sioc:Forum;
                   sioc:type "bookmark";
                   sioc:id ?forum_name;
                   sioc:has_member ?owner.
       ?owner sioc:id "kidehen".
       ?forum sioc:container_of ?post .
       ?post  dct:title ?title .
       optional { ?post sioc:link ?link  }
       optional { ?post sioc:links_to ?url }
       optional { ?post sioc:topic ?topic.
                        ?topic a skos:Concept;
                                  skos:prefLabel ?tag}.
     } 

Unmodified dump from the QBE (this will be beautified automatically in due course by the QBE):

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX sioc: <http://rdfs.org/sioc/ns#>
PREFIX dct: <http://purl.org/dc/elements/1.1/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>


SELECT ?var8 ?var9 ?var13 ?var14 ?var24 ?var27 ?var29 ?var54 ?var56
WHERE
{
graph ?graph {
 ?var8 rdf:type sioc:Forum .
 ?var8 sioc:container_of ?var9 .
 ?var8 sioc:type "bookmark" .
 ?var8 sioc:id ?var54 .
 ?var8 sioc:has_member ?var56 .
 ?var9 rdf:type sioc:Post .
 OPTIONAL {?var9 dc:title ?var13} .
 OPTIONAL {?var9 sioc:links_to ?var14} .
 OPTIONAL {?var9 sioc:link ?var29} .
 ?var9 sioc:has_creator ?var37 .
 OPTIONAL {?var9 sioc:topic ?var24} .
 ?var24 rdf:type skos:Concept .
 OPTIONAL {?var24 skos:prefLabel ?var27} .
 ?var56 rdf:type sioc:User .
 ?var56 sioc:id "kidehen" .
 }
} 

Current missing items re. Visual QBE for SPARQL are:

1. Ability to Save properly to WebDAV so that I can then expose various saved SPARQL Queries (.rq file) from my Data Space via URIs
2. Handling of DISTINCT, FILTERS (note: OPTIONAL is handled via dotted predicate-links)
3. General tidying up re. click event handling etc.

1. Export some bookmarks from your local browser to the usual HTML bookmarks dump file
2. Create an ODS-Bookmarks Instance using your new ODS account
3. Use the ODS-Bookmark Instance to import your local bookmarks from the HTML dump file
4. Repeat the same import sequence using the ODS-Bookmark Instance, but this time pick the del.icio.us option
5. Build your query (change 'kidehen' to your ODS-user-name)
6. That's it you now have Semantic Web presence in the form of a Data Space for your local and del.icio.us hosted bookmarks with tags integrated

Quick Query Builder Tip: You will need to import the following (using the Import Button in the Ontologies & Schemas side-bar);

1. http://www.w3.org/1999/02/22-rdf-syntax-ns# (RDF)
2. http://rdfs.org/sioc/ns# (SIOC)
3. http://purl.org/dc/elements/1.1/ (Dublin Core)
4. http://www.w3.org/2004/02/skos/core# (SKOS)

Browser Support: The SPARQL QBE is SVG based and currently works fine with the following browsers; Firefox 1.5/2.0, Camino (Cocoa variant of Firefox for Mac OS X), Webkit (Safari pre-release / advanced sibling), Opera 9.x. We are evaluating the use of the Adobe SVG plugin re. IE 6/7 support.

Of course this should be a screencast, but I am the middle of a plethora of things right now :-)

A quick FYI:
Virtuoso has offered a DBMS hosted Filesystem via WebDAV for a number of years, but the implications of this functionality have remained unclear for just as long. Thus, we developed (a few years ago) and released (recently) an application layer above Virtuoso's WebDAV storage realm called: “The OpenLink Briefcase” (nee. oDrive). This application allows you to view items uploaded by content type and/or kind (People, Business Cards, Calendars, Business Reports, Office Documents, Photos, Blog Posts, Feed Channels/Subscriptions, Bookmarks etc..). it also includes automatic metadata extraction (where feasible) and indexing. Naturally, as an integral part of our “OpenLink Data Spaces” (ODS) product offering, it supports GData, URIQA, SPARQL (note: WebDAV metadata is sync'ed with Virtuoso's RDF Triplestore), SQL, and WebDAV itself.

You can explore the power of this product via the following routes:

1. Download the Virtuoso Open Source Edition and the ODS add-ons or
2. Visit our live demo server (note: this is strictly a demo server with full functionality available) and simply register and then create a “Briefcase” application instance
3. Digest this Briefcase Home Page Screenshot

I added the missing piece regarding the "Virtuoso Conductor" (the Web based Admin UI for Virtuoso) to the original post below. I also added a link to our live SPARQL Demo so that anyone interested can start playing around with SPARQL and SPARQL integrated into SQL right away.

Another good thing about this post is the vast amount of valuable links that it contains. To really appreciate this point simply visit my Linkblog (excuse the current layout :-) - a Tab if you come in via the front door of this Data Space (what I used to call My Weblog Home Page).

"Free" Databases: Express vs. Open-Source RDBMSs: "Open-source relational database management systems (RDBMSs) are gaining IT mindshare at a rapid pace. As an example, BusinessWeek's February 6, 2006 ' Taking On the Database Giants ' article asks 'Can open-source upstarts compete with Oracle, IBM, and Microsoft?' and then provides the answer: 'It's an uphill battle, but customers are starting to look at the alternatives.'

There's no shortage of open-source alternatives to look at. The BusinessWeek article concentrates on MySQL, which BW says 'is trying to be the Ikea of the database world: cheap, needs some assembly, but has a sleek, modern design and does the job.' The article also discusses Postgre[SQL] and Ingres, as well as EnterpriseDB, an Oracle clone created from PostgreSQL code*. Sun includes PostgreSQL with Solaris 10 and, as of April 6, 2006, with Solaris Express.**

*Frank Batten, Jr., the investor who originally funded Red Hat, invested a reported $16 million into Great Bridge with the hope of making a business out of providing paid support to PostgreSQL users. Great Bridge stayed in business only 18 months , having missed an opportunity to sell the business to Red Hat and finding that selling $50,000-per-year support packages for an open-source database wasn't easy. As Batten concluded, 'We could not get customers to pay us big dollars for support contracts.' Perhaps EnterpriseDB will be more successful with a choice of $5,000, $3,000, or $1,000 annual support subscriptions .

**Interestingly, Oracle announced in November 2005 that Solaris 10 is 'its preferred development and deployment platform for most x64 architectures, including x64 (x86, 64-bit) AMD Opteron and Intel Xeon processor-based systems and Sun's UltraSPARC(R)-based systems.'

There is a surfeit of reviews of current MySQL, PostgreSQL and—to a lesser extent—Ingres implementations. These three open-source RDBMSs come with their own or third-party management tools. These systems compete against free versions of commercial (proprietary) databases: SQL Server 2005 Express Edition (and its MSDE 2000 and 1.0 predecessors), Oracle Database 10g Express Edition, IBM DB2 Express-C, and Sybase ASE Express Edition for Linux where database size and processor count limitations aren't important. Click here for a summary of recent InfoWorld reviews of the full versions of these four databases plus MySQL, which should be valid for Express editions also. The FTPOnline Special Report article, 'Microsoft SQL Server Turns 17,' that contains the preceding table is here (requires registration.)

SQL Server 2005 Express Edition SP-1 Advanced Features

SQL Server 2005 Express Edition with Advanced Features enhances SQL Server 2005 Express Edition (SQL Express or SSX) dramatically, so it deserves special treatment here. SQL Express gains full text indexing and now supports SQL Server Reporting Services (SSRS) on the local SSX instance. The SP-1 with Advanced Features setup package, which Microsoft released on April 18, 2006, installs the release version of SQL Server Management Studio Express (SSMSE) and the full version of Business Intelligence Development Studio (BIDS) for designing and editing SSRS reports. My 'Install SP-1 for SQL Server 2005 and Express' article for FTPOnline's SQL Server Special Report provides detailed, illustrated installation instructions for and related information about the release version of SP-1. SP-1 makes SSX the most capable of all currently available Express editions of commercial RDBMSs for Windows.

OpenLink Software's Virtuoso Open-Source Edition

OpenLink Software announced an open-source version of it's Virtuoso Universal Server commercial DBMS on April 11, 2006. On the initial date of this post, May 2, 2006, Virtuoso Open-Source Edition (VOS) was virtually under the radar as an open-source product. According to this press release, the new edition includes:

• SPARQL compliant RDF Triple Store
• SQL-200n Object-Relational Database Engine (SQL, XML, and Free Text)
• Integrated BPEL Server and Enterprise Service Bus
• WebDAV and Native File Server
• Web Application Server that supports PHP, Perl, Python, ASP.NET, JSP, etc.
• Runtime Hosting for Microsoft .NET, Mono, and Java

VOS only lacks the virtual server and replication features that are offered by the commercial edition. VOS includes a Web-based administration tool called the "Virtuoso Conductor" According to Kingsley Idehen's Weblog, 'The Virtuoso build scripts have been successfully tested on Mac OS X (Universal Binary Target), Linux, FreeBSD, and Solaris (AIX, HP-UX, and True64 UNIX will follow soon). A Windows Visual Studio project file is also in the works (ETA some time this week).'

InfoWorld's Jon Udell has tracked Virtuoso's progress since 2002, with an additional article in 2003 and a one-hour podcast with Kingsley Idehen on April 26, 2006. A major talking point for Virtuoso is its support for Atom 0.3 syndication and publication, Atom 1.0 syndication and (forthcoming) publication, and future support for Google's GData protocol, as mentioned in this Idehen post. Yahoo!'s Jeremy Zawodny points out that the 'fingerprints' of Adam Bosworth, Google's VP of Engineering and the primary force behind the development of Microsoft Access, 'are all over GData.' Click here to display a list of all OakLeaf posts that mention Adam Bosworth.

One application for the GData protocol is querying and updating the Google Base database independently of the Google Web client, as mentioned by Jeremy: 'It's not about building an easier onramp to Google Base. ... Well, it is. But, again, that's the small stuff.' Click here for a list of posts about my experiences with Google Base. Watch for a future OakLeaf post on the subject as the GData APIs gain ground.

Open-Source and Free Embedded Database Contenders

Open-source and free embedded SQL databases are gaining importance as the number and types of mobile devices and OSs proliferate. Embedded databases usually consist of Java classes or Windows DLLs that are designed to minimize file size and memory consumption. Embedded databases avoid the installation hassles, heavy resource usage and maintenance cost associated with client/server RDBMSs that run as an operating system service.

Andrew Hudson's December 2005 'Open Source databases rounded up and rodeoed' review for The Enquirer provides brief descriptions of one commercial and eight open source database purveyors/products: Sleepycat, MySQL, PostgreSQL, Ingres, InnoBase, Firebird, IBM Cloudscape (a.k.a, Derby), Genezzo, and Oracle. Oracle Sleepycat* isn't an SQL Database, Oracle InnoDB* is an OEM database engine that's used by MySQL, and Genezzo is a multi-user, multi-server distributed database engine written in Perl. These special-purpose databases are beyond the scope of this post.

* Oracle purchased Sleepycat Software, Inc. in February 2006 and purchased Innobase OY in October 2005 . The press release states: 'Oracle intends to continue developing the InnoDB technology and expand our commitment to open source software.'

Derby is an open-source release by the Apache Software Foundation of the Cloudscape Java-based database that IBM acquired when it bought Informix in 2001. IBM offers a commercial release of Derby as IBM Cloudscape 10.1. Derby is a Java class library that has a relatively light footprint (2 MB), which make it suitable for client/server synchronization with the IBM DB2 Everyplace Sync Server in mobile applications. The IBM DB2 Everyplace Express Edition isn't open source or free*, so it doesn't qualify for this post. The same is true for the corresponding Sybase SQL Anywhere components.**

* IBM DB2 Everyplace Express Edition with synchronization costs $379 per server (up to two processors) and $79 per user. DB2 Everyplace Database Edition (without DB2 synchronization) is $49 per user. (Prices are based on those when IBM announced version 8 in November 2003.)

** Sybase's iAnywhere subsidiary calls SQL Anywhere 'the industry's leading mobile database.' A Sybase SQL Anywhere Personal DB seat license with synchronization to SQL Anywhere Server is $119; the cost without synchronization wasn't available from the Sybase Web site. Sybase SQL Anywhere and IBM DB2 Everyplace perform similar replication functions.

Sun's Java DB, another commercial version of Derby, comes with the Solaris Enterprise Edition, which bundles Solaris 10, the Java Enterprise System, developer tools, desktop infrastructure and N1 management software. A recent Between the Lines blog entry by ZDNet's David Berlind waxes enthusiastic over the use of Java DB embedded in a browser to provide offline persistence. RedMonk analyst James Governor and eWeek's Lisa Vaas wrote about the use of Java DB as a local data store when Tim Bray announced Sun's Derby derivative and Francois Orsini demonstrated Java DB embedded in the Firefox browser at the ApacheCon 2005 conference.

Firebird is derived from Borland's InterBase 6.0 code, the first commercial relational database management system (RDBMS) to be released as open source. Firebird has excellent support for SQL-92 and comes in three versions: Classic, SuperServer and Embedded for Windows, Linux, Solaris, HP-UX, FreeBSD and MacOS X. The embedded version has a 1.4-MB footprint. Release Candidate 1 for Firebird 2.0 became available on March 30, 2006 and is a major improvement over earlier versions. Borland continues to promote InterBase, now at version 7.5, as a small-footprint, embedded database with commercial Server and Client licenses.

SQLite is a featherweight C library for an embedded database that implements most SQL-92 entry- and transitional-level requirements (some through the JDBC driver) and supports transactions within a tiny 250-KB code footprint. Wrappers support a multitude of languages and operating systems, including Windows CE, SmartPhone, Windows Mobile, and Win32. SQLite's primary SQL-92 limitations are lack of nested transactions, inability to alter a table design once committed (other than with RENAME TABLE and ADD COLUMN operations), and foreign-key constraints. SQLite provides read-only views, triggers, and 256-bit encryption of database files. A downside is the the entire database file is locked when while a transaction is in progress. SQLite uses file access permissions in lieu of GRANT and REVOKE commands. Using SQLite involves no license; its code is entirely in the public domain.

The Mozilla Foundation's Unified Storage wiki says this about SQLite: 'SQLite will be the back end for the unified store [for Firefox]. Because it implements a SQL engine, we get querying 'for free', without having to invent our own query language or query execution system. Its code-size footprint is moderate (250k), but it will hopefully simplify much existing code so that the net code-size change should be smaller. It has exceptional performance, and supports concurrent access to the database. Finally, it is released into the public domain, meaning that we will have no licensing issues.'

Vieka Technology, Inc.'s eSQL 2.11 is a port of SQLite to Windows Mobile (Pocket PC and Smartphone) and Win32, and includes development tools for Windows devices and PCs, as well as a .NET native data provider. A conventional ODBC driver also is available. eSQL for Windows (Win32) is free for personal and commercial use; eSQL for Windows Mobile requires a license for commercial (for-profit or business) use.

HSQLDB isn't on most reviewers' radar, which is surprising because it's the default database for OpenOffice.org (OOo) 2.0's Base suite member. HSQLDB 1.8.0.1 is an open-source (BSD license) Java dembedded database engine based on Thomas Mueller's original Hypersonic SQL Project. Using OOo's Base feature requires installing the Java 2.0 Runtime Engine (which is not open-source) or the presence of an alternative open-source engine, such as Kaffe. My prior posts about OOo Base and HSQLDB are here, here and here.

The HSQLDB 1.8.0 documentation on SourceForge states the following regarding SQL-92 and later conformance:

HSQLDB 1.8.0 supports the dialect of SQL defined by SQL standards 92, 99 and 2003. This means where a feature of the standard is supported, e.g. left outer join, the syntax is that specified by the standard text. Many features of SQL92 and 99 up to Advanced Level are supported and here is support for most of SQL 2003 Foundation and several optional features of this standard. However, certain features of the Standards are not supported so no claim is made for full support of any level of the standards.

Other less well-known embedded databases designed for or suited to mobile deployment are Mimer SQL Mobile and VistaDB 2.1 . Neither product is open-source and require paid licensing; VistaDB requires a small up-front payment by developers but offers royalty-free distribution.

Java DB, Firebird embedded, SQLite and eSQL 2.11 are contenders for lightweight PC and mobile device database projects that aren't Windows-only.

SQL Server 2005 Everywhere

If you're a Windows developer, SQL Server Mobile is the logical embedded database choice for mobile applications for Pocket PCs and Smartphones. Microsoft's April 19, 2006 press release delivered the news that SQL Server 2005 Mobile Editon (SQL Mobile or SSM) would gain a big brother—SQL Server 2005 Everywhere Edition.

Currently, the SSM client is licensed (at no charge) to run in production on devices with Windows CE 5.0, Windows Mobile 2003 for Pocket PC or Windows Mobile 5.0, or on PCs with Windows XP Tablet Edition only. SSM also is licensed for development purposes on PCs running Visual Studio 2005. Smart Device replication with SQL Server 2000 SP3 and later databases has been the most common application so far for SSM.

By the end of 2006, Microsoft will license SSE for use on all PCs running any Win32 version or the preceding device OSs. A version of SQL Server Management Studio Express (SSMSE)—updated to support SSE—is expected to release by the end of the year. These features will qualify SSE as the universal embedded database for Windows client and smart-device applications.

For more details on SSE, read John Galloway's April 11, 2006 blog post and my 'SQL Server 2005 Mobile Goes Everywhere' article for the FTPOnline Special Report on SQL Server."

(Via OakLeaf Systems.)

What Is Virtuoso?

A powerful next generation server product that implements otherwise distinct server functionality within a single server product. Think of Virtuoso as the server software analog of a dual core processor where each core represents a traditional server functionality realm.

Where did it come from?

The Virtuoso History page tells the whole story.

What Functionality Does It Provide?

1. Object-Relational DBMS Engine (ORDBMS like PostgreSQL and DBMS engine like MySQL)

2. XML Data Management (with support for XQuery, XPath, XSLT, and XML Schema)

3. RDF Triple Store (or Database) that supports SPARQL (Query Language, Transport Protocol, and XML Results Serialization format)

4. Service Oriented Architecture (it combines a BPEL Engine with an ESB)

5. Web Application Server (supports HTTP/WebDAV)

6. NNTP compliant Discussion Server

90% of the aforementioned functionality has been available in Virtuoso since 2000 with the RDF Triple Store being the only 2006 item.

What Platforms are Supported

The Virtuoso build scripts have been successfully tested on Mac OS X (Universal Binary Target), Linux, FreeBSD, and Solaris (AIX, HP-UX, and True64 UNIX will follow soon). A Windows Visual Studio project file is also in the works (ETA some time this week).

Why Open Source?

Simple, there is no value in a product of this magnitude remaining the "best kept secret". That status works well for our competitors, but absolutely works against the legions of new generation developers, systems integrators, and knowledge workers that need to be aware of what is actually achievable today with the right server architecture.

What Open Source License is it under?

GPL version 2.

What's the business model?

Dual licensing.

The Open Source version of Virtuoso includes all of the functionality listed above. While the Virtual Database (distributed heterogeneous join engine) and Replication Engine (across heterogeneous data sources) functionality will only be available in the commercial version.

Where is the Project Hosted?

On SourceForge.

Is there a product Blog?

Of course!

Up until this point, the Virtuoso Product Blog has been a covert live demonstration of some aspects of Virtuoso (Content Management). My Personal Blog and the Virtuoso Product Blog are actual Virtuoso instances, and have been so since I started blogging in 2003.

Is There a product Wiki?

Sure! The Virtuoso Product Wiki is also an instance of Virtuoso demonstrating another aspect of the Content Management prowess of Virtuoso.

What About Online Documentation?

Yep! Virtuoso Online Documentation is hosted via yet another Virtuoso instance. This particular instance also attempts to demonstrate Free Text search combined with the ability to repurpose well formed content in a myriad of forms (Atom, RSS, RDF, OPML, and OCS).

What about Tutorials and Demos?

The Virtuoso Online Tutorial Site has operated as a live demonstration and tutorial portal for a numbers of years. During the same timeframe (circa. 2001) we also assembled a few Screencast style demos (their look feel certainly show their age; updates are in the works).

BTW - We have also updated the Virtuoso FAQ and also released a number of missing Virtuoso White Papers (amongst many long overdue action items).

"...Also today I came across the latest project of a man who wants to tear down Tim Berners-Lee's World Wide Web and replace it with his own vision. It used to be known as Xanadu, but has since morphed into Transliterature, A Humanist Design. I am of course referring to Ted Nelson, who invented the term 'hypertext' in 1965 and is generally regarded as a computing pioneer.

Ted Nelson recently wrote an essay about 'Indirect Documents', which got Slashdotted today. In the essay Nelson outlines why (in his opinion) the Xanadu project failed and he explains his new vision for Transliterature. He takes a number of potshots at Tim Berners-Lee's WWW on the way, e.g.:

'Why don't I like the web? I hate its flapping and screeching and emphasis on appearance; its paper-simulation rectangles of Valuable Real Estate, artifically created by the NCSA browser, now hired out to advertisers; its hierarchies exposed and imposed; its untyped one-way links only from inside the document. (The one-way links hidden under text were a regrettable simplification of hypertext which I assented to in '68 on the HES project. But that's another story.) Only trivial links are possible; there is nothing to support careful annotation and study; and, of course, there is no transclusion.'

Ted Nelson is certainly an original and I'm glad he's still around to throw spanners in the works. I've written about him before and I'm sure I will again, Web 2.0 or not.

(Excerpted From: Read/Write Web.)

My thoughts on the commentary above:

There is nothing fundamentally incompatible between Ted Nelson's pursuits and future incarnation's of the Web. None whatsoever -- we are simply working our way through an process. The process in question is what I call "standards driven ubiquity" (becoming de facto at Internet Speed). Remember Sun's "The Network is the Computer" vision? Well, without a "Computer" in mind-space you can't think in terms of "Operating Systems". Thats all changing, because today we are gradually beginning to accept the imminent reality that "The Internet is the Operating System" and not Windows/UNIX/Mac OS X/Others. Ahem! And after the Operating System what comes next? I think a set of Application Programming Interfaces (APIs), and I think we know what that is (in all of its controversial glory), the very thing we refer to as Web 2.0 (the APIs for the Internet Operating System).

Note: In addition to the Computer, Operating System, and Application Programming Interfaces, we also have those frequently misunderstood and under-appreciated workhorses called "Databases" in place (but we still call them Web Sites for now). And by the way, "Internet Filesystem" has been there forever, but for some reason we can't see WebDAV in all its current and future glory (that will change very soon also!).

Ted and TBL are cool with each (whether they know it or not)! I see no mutual exclusivity in their collective visions (IMHO) :-)

Here goes:

Blog Editing

I can use any editor that supports the following Blog Post APIs:

- Moveable Type

- Meta Weblog

- Blogger

Typically I use Virtuoso (which has an unreleased WYSIWYG blog post editor), Newzcrawler, ecto, Zempt, or w.bloggar for my posts. If a post is of interest to me, or relevant to our company or customers I tend to perform one of the following tasks:

- Generate a post using the "Blog This" feature of my blog editor

- Write a new post that was triggered by a previously read post etc.

Either way, the posts end up in our company wide blog server that is Virtuoso based (more about this below). The internal blog server automatically categorizes my blog posts, and automagically determines which posts to upstream to other public blogs that I author (e.g http://kidehen.typepad.com ) or co-author (e.g http://www.openlinksw.com/weblogs/uda and http://www.openlinksw.com/weblogs/virtuoso ). I write once and my posts are dispatched conditionally to multiple outlets.

RSS/Atom/RDF Aggregation & Reading

I discover, subscribe to, and view blog feeds using Newzcrawler (primarily), and from time to time for experimentation and evaluation purposes I use RSS Bandit, FeedDemon, and Bloglines. I am in the process of moving this activity over to Virtuoso completely due to the large number of feeds that I consume on a daily basis (scalability is a bit of a problem with current aggregators).

Blog Publishing

When you visit my blog you are experiencing the  soon to be released Virtuoso Blog Publishing engine first hand, which is how WebDAV, SQLX, XQuery/XPath, and Free Text etc. come into the mix.

Each time I create a post internally, or subscribe to an external feed, the data ends up in Virtuoso's SQL Engine (this is how we handle some of the obvious scalability challenges associated with large subscription counts). This engine is SQL2000N based, which implies that it can transform SQL to XML on the fly using recent extensions to SQL in the form of SQLX (prior to the emergence of this standard we used the FOR XML SQL syntax extensions for the same result). It also has its own in-built XSLT processor (DB Engine resident), and validating XML parser (with support for XML Schema).  Thus, my RSS/RDF/Atom archives, FOAF, BlogRoll, OPML, and OCS blog syndication gems are all live examples of SQLX documents that leverage Virtuoso's WebDAV engine for exposure to Blog Clients.

Blog Search

When you search for blog posts using the basic or advanced search features of my blog, you end up interacting with one of the following methods of querying data hosted in Virtuoso: Free Text Search, XPath, or XQuery. The result sets produced by the search feature uses SQLX to produce subscription gems (RSS/Atom/RDF/OpenSearch) and URIs that enable dynamic tracking of my posts using your search keywords.

BTW - the http://www.openlinksw.com/blog/~kidehen blog home page exists as a result of Virtuoso's Virtual Domain / Multi-Homing Web Server functionality. The entire site resides in an Object Relational DBMS, and I can take my DB file across Windows, Solaris, Linux, Mac OS X, FreeBSD, AIX, HP-UX, IRIX, and SCO UnixWare without missing a single beat! All I have to do is instantiate my Virtuoso server and my weblog is live.

Email As A Platform It looks like more people are starting to realize that email is more than it seems. Especially given the drastic increase in storage size of web-based email applications, more people are realizing that email is basically a personal database. People simply store information in their email, from contact information that was emailed to them to schedule information to purchase tracking from emailed receipts. Lots of people email messages to themselves, realizing that email is basically the best "permanent" filing system they have. That's part of the reason why good email search is so important. Of course, what the article doesn't discuss is the next stage of this evolution. If you have a database of important information, the next step is to build useful applications on top of it. In other words, people are starting to realize that email, itself, is a platform for personal information management.

Wanted: Wiki + WebDAV = WikiFolders In my opinion the two greatest advancements in web technologies in recent years have been WikiWikiWeb and WebDAV. Firstly, two predictions: Within the next 3 years more documents worldwide will be transferred and accessed via WebDAV than FTP, NFS, and......

Stickiness is a defining characteristic of Web 1.0 . It's all about eyeballs (site visitors) which implied ultimately that all early Web business models ended up down the advertising route.

I always felt that Web 1.0 was akin to having a crowd of people at your reception area seeking a look at your corporate brochures, and then someone realizes that you could start selling AD space in these brochures in response to the growing crowd size and frequency of congregation. The long-term folly of this approach is now obvious, as many organizations forgot their core value propositions (expressed via product offerings) in the process and wandered blindly down the AD model cul-de-sac, and we all know what happened down there..

Web 2.0 is taking shape (the inflection is in its latter stages), and the defining characteristics of Web 2.0 are:

1. Fabric of Executable Endpoints
   
2. Semantic Content (the RSS/RDF/Atom/FOAF semantic crumbs emerging from the Blogosphere are great examples of things to come re. XQuery queries over HTTP for instance) Migration from the Web Site (defined by static or dynamic HTML page generation) concept, to that of a "Web Point of Presence" (I don't know if this term will catch on, but the conceptual essence here is factual) that enables an organization to achieve the following:
   
• Package/catalog value proposition (product and services) using RSS/RDF/Atom
  
• Provide SOAP compliant Executable Endpoints (Web Services) for consuming value proposition (as opposed to being distracted by the AD model)
  
• Provide Web Services for consummating contracts associated with core value proposition Identification of internal efficiencies, new products/services that leverage Semantic Content and Web Services, and tangibly exploit:
  
• Composite Web Services construction from legacy monolithic application pools
  
• Standards based (e.g. BPEL) orchestration and integration of disparate composite services (across the Fabric referred to above)

When you factor in all of the above, the real question is whether Google and others are equipped to exploit Web 2.0?  To some degree, is the best answer at the current time as they have commenced the transition from "content only" web site to web platform (via the many Web Services initiatives that expose SOAP and REST interfaces to various services), but there is much more to this journey, and that's the devil in the "competitive landscape details".

From my obviously biased perspective, I think Virtuoso and Yukon+WinFS provide the server models for driving Web 2.0 points of presence (single server instances that implement multiple protocols). Thus, if Google, Yahoo! et al. aren't exploiting these or similar products, then they will be vulnerable over the long term to the competitve challenges that a Web 2.0 landscape will present.

How secure is your data? Looking at your information management resources through a would-be intruder's eyes can help you find (and fix) vulnerabilities.

Naturally :-)

When E. F. Codd developed his relational data model in 1970, the business world was a different place. Almost 35 years after his seminal work appeared, RDBMSs that sprung from Codd's ideas are the standard for storing corporate information. And, with government and industry regulations dictating what kinds of information companies have to store, manage, and audit (and for how long), protecting this information is more important than ever. Unfortunately, it's also more challenging

Even in 1985, when Dr. Codd published 12 guidelines for RDBMSs, there was little concern for data security. In those days, gaining access to a database was so difficult that advanced security features on the database were irrelevant.

Today, RDBMSs carry the lifeblood of every organization. Note the use of the plural: Organizations now have many databases that are decentralized in terms of use and security controls. E-business demands that data access be extended to customers, partners, suppliers, and other parties who were rarely considered in the early data management days. With all this availability ? not to mention pressure from an array of government and industry regulations (see the sidebar, "Security and Compliance") ? the need to control exactly who can access or modify data is becoming paramount.

Absolute facts, that are still partially understood at best. For instance we are still in a so called "Information Age" in which standards based data access remains an issue of contempt instead of absolute necessity.

There are a number of prevailing myths about standards based data access that continue to cloak reality:

1. ODBC, JDBC, ADO.NET, OLEDB all deliver poor performance (compared to their native, proprietary, and database specific counterparts; native interfaces)
   
2. You can't really right generic database applications with these standards due to inconsistencies in the DBMS implementations of SQL (not true! there are many aspects of the specs that address these concerns if only a majority of driver vendors would implement these features, and the application developers actually used them by seeking drivers with full implementations).

Even if the above were true (which I refute strongly), how about the general security vulnerabilities that affect both Native, and Standards compliant, data access interfaces?

Aaron's article does a good job of highlighting 6 areas of vulnerability:

1. DBMS Defaults (usernames and passwords)
2. Authentication (at connect time)
3. Database Privileges
4. Fixpaks
5. Buffer Overflows
6. SQL Injection

What I have been able to do very quickly (thanks to blogging, and the power of a blog engine that supports WebDAV), is write a tabulated response to each of the items (bar Fixpaks) indicating how the OpenLink Multi-Tier Data Access Drivers (for ODBC, JDBC, ADO.NET, and OLEDB) protect corporate databases from each of these vulnerabilities.

To cut a long story short, we are increasingly living a contradiction where the terms "simple" and "free" are supposed to lead us to products that can adequately handle the challenges of an increasingly sophisticated grid of inter-connecting point.

I have been asked on numerous occassions, "How can you build a company and business based on data access technology?". My reply is the same as usual, "because everything comes down to data". If the data is compromised in anyway, then kiss Information, Knowledge, and everything else goodbye!

This issue really brings WebDAV into scope as this is the protocol that enables this capability (as covered by Jon's piece), and it is one of the many client and server side protocols implemented by OpenLink Virtuoso (the key to how Virtuoso delivers URI based SQL-XML, XQuery, XPath services).

When you install Virtuoso you simply have to start the Virtuoso server instance to the get WebDAV functionality going. All of Virtuoso's services are advertised at ports, and in the case of WebDAV you will find this at port 8890 if you start the demo database.

To exploit the Virtuoso/WebDAV server from any WebDAV client (or point urls at WebDAV hosted resources) simply do the following:

1. Install Virtuoso and depending on your OS do the following:
• Windows - create a Web Folder that points to a WebDAV server
• Mac OS X - mount a WebDAV folder
• Linux - mount a WebDAV directory (also see the Davfs2 Open Source project)
• You can also make WebDAV client calls from Virtuoso's Stored Procedure Language (Virtuoso PL) or use WebDAV implementations in any development environment of your choice (.NET, Java, .
  
2. Place content that you want to reference in your mails in your WebDAV repository via any of the client side mechanisms described in step 1. You can see the results of this in my earlier blog post, even better pass the url on in an email! Or browse the WebDAV folder (there are some nuggets deliberately left in place :-) )
   
• You could simply save an Office Doc (powerpoint, excel, word etc) to this location and the circulate urls in your mails (this has been standard practice at OpenLink for many years; we even have a full blown portal server that would soon be available as a public service to sharing anything via DAV and as usual some more... stay tuned)
  
3. That's it for any platform (Windows, Linux, Mac OS X, FreeBSD, Solaris, AIX, HP-UX etc.) once you install Virtuoso!

BTW - This blog is WebDAV based (it's a live instance of Virtuoso doing many things; WebDAV, HTTP, SQL-XML based feed generation for ATOM, RSS, Blog Post APIs support (Moveable Type, Metaweblog, Blogger, ATOM), Free Text, XPath, XQuery, and more). 

Mozilla's SQL Support allows applications to directly connect to SQL databases. A web application no longer needs to pass information through a scripting language, such as Perl or Python, in order to recieve information it can use. The removal of the layer seperating applications and data simplifies the job of the programmer.

Somehow I missed this effort, and only stumbled across it today after experimenting with Virtuoso's SyncML features (and then pondering about OutLook, WinFS, and what may or may not happen with SyncML support - another story).

As usual the SQL binding to Mozilla caught my attention (I do recall trying to get Marc and Jim Clark to head down this path many years ago via an email; at least Jim acknowledged not knowing that much about SQL and past it on.., and as for Marc well... nothing happened).

A few

As I completed this post a bell went off! Why not use this for a quick live demo of Virtuoso's hosting capabilities? Starting off with something simple like PHP for instance?

So, I quickly did the following:

1. Cut and pasted the PHP version of this programming puzzle into a text file (using notepad)
2. Copy and pasted from my Windows Directory to my Virtuoso WebDAV directories on our Windows and Linux Virtuoso Demo Servers. I achieved this by creating Web Folders (Windows OS level

To share your calendars, you need access to a webDAV server. If you run your own web server, you can install mod_dav, a free Apache module that will turn your web server into a webDAV server. Instructions on how to set it up are on their website. Once you set up your webDAV server, you can publish your calendar to the site, then subscribe to it from any other Mozilla Calendar. Automatically updating the calendar will give you a poor man's calendar server.

Through WebDAV we will be able to share calendars across disparate calendaring tools (albeit with some degree of pain when Outlook is in the mix). Even better for me, I can post my shared calendar data via a Virtuoso instance (internally and externally since WebDAV is one of the many protocols that it implements), in short I could even seriously consider generating this on the fly and sharing it via this blog (Wow!).

We aren't too many miles away from open and standards compliant Unified Data Storage thanks to WebDAV.

Databases get a grip on XML
From Inforworld.

The next iteration of the SQL standard was supposed to arrive in 2003. But SQL standardization has always been a glacially slow process, so nobody should be surprised that SQL:2003 ? now known as SQL:200n ? isn?t ready yet. Even so, 2003 was a year in which XML-oriented data management, one of the areas addressed by the forthcoming standard, showed up on more and more developers? radar screens.  >> READ MORE

This article rounds up product for 2003 in the critical area of Enterprise Database Technology. It's certainly provides an apt reflection of how Virtuoso compares with offerings from some the larger (but certainly slower to implement) database vendors in this space. As usual Jon Udell's quote pretty much sums this up:

"While the spotlight shone on the heavyweight contenders, a couple of agile innovators made noteworthy advances in 2003. OpenLink Software?s Virtuoso 3.0, which we reviewed in March, stole thunder from all three major players. Like Oracle, it offers a WebDAV-accessible XML repository. Like DB2 Information Integrator, it functions as database middleware that can perform federated ?joins? across SQL and XML sources. And like the forthcoming Yukon, it embeds the .Net CLR (Common Language Runtime), or in the case of Linux, Novell/Ximian?s Mono."

Albeit still somewhat unknown to the broader industry we have remained true our "innovator" discipline, which still remains our chosen path to market leadership. Thus, its worth a quick Virtuoso release history, and features recap as we get set to up the ante even further in 2004:

1998 - Virtuoso's initial public beta release with functional emphasis on Virtual Database Engine for ODBC and JDBC Data Sources.

1999 - Virtuoso's official commercial release, with emphasis still on Virtual Database functionality for ODBC, JDBC accessible SQL Databases.

2000 - Virtuoso 2.0 adds XML Storage, XPath, XML Schema, XQuery, XSL-T, WebDAV, SOAP, UDDI, HTTP, Replication, Free Text Indexing (*feature update*), POP3, and NNTP support.

2002 - Virtuoso 2.7 extends Virtualization prowess beyond data access via enhancements to its Web Services protocol stack implementation by enabling SQL Stored Procedures to be published as Web Services. It also debuts its Object-Relational engine enhancements that include the incorporation of Java and Microsoft .NET Objects into its User Defined Type, User Defined Functions, and Stored Procedure offerings.

2003 - Virtuoso 3.0 extends data and application logic virtualization into the Application Server realm (basically a Virtual Application server too!), by adding support for ASP.NET, PHP, Java Server Pages runtime hosting (making applications built using any of these languages deployable using Virtuoso across all supported platforms).

Collectively each of these releases have contributed to a very premeditated architecture and vision that will ultimately unveil the inherent power of critical I.S infrastructure virtualization along the following lines; data storage, data access , and application logic via coherent integration of SQL, XML, Web Services, and Persistent Stored Modules (.NET, Java, and other object based component building blocks).

 

It starts here with one of many excerpts from Scoble's blog:

Dave Winer, Jon Udell, and now Gerald Bauer says that Microsoft is killing the Web. Or trying to.

The guys above are pretty seasoned individuals (they save me a lot of writing too amongst other things).

Now here is a response from Microsoft?s Blog evangelist supremo Scoble to their comments and genuine concerns.

OK, let's assume that's true.

Microsoft has 55,000 employees. $50 billion or so in the bank.

Yet what has gotten me to use the Web less and less lately? RSS 2.0.

Seriously. I rarely use the browser anymore (except to post my weblog since I use Radio UserLand).

See the irony there? Dave Winer (who at minimum popularized RSS 2.0) has done more to get me to move away from the Web than a huge international corporation that's supposedly focused on killing the Web.

Now, let's look at what's really going on here. We're going back to being a great platform company. We're trying to provide a platform that lets developers build new applications that are impossible to build on other platforms. At the PDC you saw some of that. New kinds of forms. New kinds of games. New kinds of business apps. New kinds of experiences.

But, we also are looking for ways to make the Web better too. Now, we haven't talked about what we're doing with the browser. I hear that'll come later. Astute Longhorn testers have already seen that we snuck a pop-up ad blocker into the browser without telling anyone about it. Whoa. That means we're gonna turn off MSN's capabilities of selling popup ads.

I hear there's more coming too. But, why should we do it all? Wasn't the point of the past four years to get Microsoft to stop trying to do it all? The DOJ and now the European Union are still after us cause we tried to do it all. Instead, let's just go back and be a great platform company.

We just gave you a great foundation for a killer new kind of application. One that goes FAR beyond HTML. And, even if you stick with Mozilla, your experiences on Longhorn will get better. For instance, fonts are being rendered in the GPU now on Longhorn. Your Web pages will look better and behave better on Longhorn than they will on any other platform. Period.

And wait until Mozilla's and other developers start exploiting things like WinFS to give you new features that display Internet-based information in whole new ways.

If Microsoft really wants to create a better platform shouldn?t this be truly futuristic? If so, then it should issue the first major salvo by dropping the restrictions on Rotor?

We are moving into the distributed component based computing age where runtime environments (.NET CLR, Mono, J2EE, and others) act a Component Execution Junction boxes (instead of the Monolithic Operating Systems of today) in a continuum of services orchestrated by messages in response to events emanating from value consumption requests (what we call application behviour today) from a myriad of value consumers (application users).

There is no need for covert and protracted protection of an obsolete Windows Operating System (the underlying fear that keeps Rotor shackled in my opinion), since its obsolescence is in full motion as Longhorn clearly demonstrates.

Imagine a fusion of sorts across Microsoft .NET, Mono, and Rotor, with a single portable runtime as the end product (slotting nicely into its place in the imminent distributed component and services era). All the benefits of programming language independence in true glory - the ECMA-CLI is all about programming language independence. Now that would be unequivocally revolutionary, and Microsoft would actually be doing what I think it has been desperately trying to achieve for a long time; the delivery of really cool technology that seriously impact us all in a positive way without the usual World Domination Concerns. 

Anyway, back to the current reality where we have covert attempts to lock us all into Windows getting more and more transparent per technology release cycle. The very antithesis of what I espoused in the last paragraph (or dream). I believe that Scoble's instincts lie in this realm too, and you never know this evangelist may turn Messiah :-)

Here's the final excerpt from Scoble?s post:

There's a whole lot of more useful stuff coming. Both for the Web and for newer Internet-centric rich-client approaches. Personally, it's about time. I'm already using the Web less and less thanks to things like RSS 2.0.

I'm watching 636 sites every day. Try to do THAT in your Web browser.

So, yes, blame it on me. I'm trying to kill the Web. Isn't it time to move on? Didn't we move on from the Apple II? Didn't we move on from DOS? Didn't we move on from Windows 3.11? Can't you see a day when we move on from the Web and get something even more fantastic? I can. Dave Winer can. Why not you? [via The Scobleizer Weblog]

If you kill the Web en route to getting us a Portable Execution Junction box from Microsoft, I think you would have served mankind pretty damned well. We won't have to gripe about Web 1.0 (Browser Driven Web) because we would be well into Web 2.0 and beyond (which doesn?t define the Web experience predominantly via browsing).

1. Third Party integration points are clearly identified
   
2. No mention of SyncML (although worst case this could be bootstrapped by a third party SyncML Adapter).

I hope other diagrams will be are clear as this, especially the ones relating to actual storage :-)

Reading the Longhorn SDK docs is a disorienting experience. Everything's familiar but different. Consider these three examples:

[Full story: Replace and defend via Jon's Radio]

"Replace & Defend" is certainly a strategy that would have awakened the entire non Microsoft Developer world during the recent PDC event. I know these events are all about preaching to the choir (Windows only developers), but as someone who has worked with Microsoft technologies as an ISV since the late 80's there is something about this events announcements that leave me concerned.

Ironically these concerns aren't about the competitive aspects of their technology disruptions, but more along the lines of how Microsoft (I hope inadvertently) generates the kinds of sentiments echoed in the comments thread from Scobles recent "How to hate Microsoft" post. As indicated in my response to this post, I don't believe Microsoft is as bad or evil as is instinctively assumed in many quarters, but I can certainly understand why they are hated by others which is really unfortunate, especially bearing in mind that they have done more good than harm to date (in my humble opinion) .

Anyway, back to my concerns post PDC which I break down as follows:

1. Disruptive assaults on existing standards with the only benefit being Microsoft platform centricity. Jon Udell addressed this in his "Replace and Defend" post (which kicked of this post), and I see exactly what he sees here, and I don't see any reason for this approach whatsoever. Even if one of these standards was deficient what stops the Microsoft from addressing these deficiencies, and then should the W3C's standards acceptance and ratification process bogs things down at least let the industry know you gave it openness a chance but have to move on etc..
   
   
2. Gradual obsolescence of existing Microsoft standards which used to provide interfaces for 3rd party ISV partners, and replacing these with totally closed infrastructure implementations that bind to Microsoft products only.  A good example is WinFS, I believe in the unified data storage concept, it's a vision that I've believed in for many years, but there is no notion from any PDC presentation or Blog that I have read so far (I aggregate a serious number of feeds) that Microsoft is committed to an architectural strategy that enables 3rd party ISVs to hook their data stores and data sources into this storage infrastructure - it's simply about Yukon (SQL Server) and that's basically it.

WinFS needs to architecturally separate the System Provider from the Data Provider (pretty much the OLE-DB architecture) with Microsoft naturally providing reference System Provider (pretty much what was demonstrated at PDC) and Data Provider (ADO.NET, OLE DB, and ODBC) implementations. Third parties can choose to produce custom WinFS Service or Data Providers which serve their data access needs. It's impractical to want to force every non SQL Server customer over to SQL Server in order them to exploit WinFS, and I certainly hope this isn't the definitive strategy at Microsoft.

Every year, as new hard disks get bigger and faster, applications catch up by producing more data. Hard disks are commonly used to store personal information: correspondence, personal contacts, and work documents. These items are currently treated as separate entities, yet they are interrelated on some level; and it's no surprise that e-mail comes from your personal contacts list and influences the work that you should be doing and hence determines the documents that you'll create. When you have a large number of items, it is important to have a flexible and efficient mechanism to search for particular items based on their properties and content. Up until now, storage mechanisms like Outlook

NETWORK WORLD NEWSLETTER: MARK GIBBS ON WEB APPLICATIONS

Today's focus: A Virtuoso of a server

By Mark Gibbs

One of the bigger drags of Web applications development is that building a system of even modest complexity is a lot like herding cats - you need a database, an applications server, an XML engine, etc., etc. And as they all come from different vendors you are faced with solving the constellation of integration issues that inevitably arise.

If you are lucky, your integration results in a smoothly functioning system. If not, you have a lot of spare parts flying in loose formation with the risk of a crash and burn at any moment.

An alternative is to look for all of these features and services in a single package but you'll find few choices in this arena.

One that is available and looks very promising is OpenLink's Virtuoso (see links below).

Virtuoso is described as a cross platform (runs on Windows, all Unix flavors, Linux, and Mac OS X) universal server that provides databases, XML services, a Web application server and supporting services all in a single package.

OpenLink's list of supported standards is impressive and includes .Net, Mono, J2EE, XML Web Services (Simple Object Application Protocol, Web Services Description Language, WS-Security, Universal Description, Discovery and Integration), XML, XPath, XQuery, XSL-T, WebDav, HTTP, SMTP, LDAP, POP3, SQL-92, ODBC, JDBC and OLE-DB.

Virtuoso provides an HTTP-compliant Web Server; native XML document creation, storage and management; a Web services platform for creation, hosting and consumption of Web services; content replication and synchronization services; free text index server, mail delivery and storage and an NNTP server.

Another interesting feature is that with Virtuoso you can create Web services from existing SQL Stored Procedures, Java classes,

C++ classes, and 'C' functions as well as create dynamic XML

documents from ODBC and JDBC data sources.

This is an enormous product and implies a serious commitment on the part of adopters due to its scope and range of services.

Virtuoso is enormous by virtue of its architectural ambitions, but actual disk requirements are

Tim O'Reilly wrote some thoughts about network aware software. Good sumup and nice ideas, why not only blogs should be net-aware (and where even blogs can be improved ;) )

"For the desktop, my personal vision is to see existing software instrumented to become increasingly web aware. It seems that Apple are doing a good job with this. (What does web aware mean for me? Being able to grok URIs, speaking WebDAV, and using open standard data formats.)" -- Edd Dumbill

[via Bitflux Blog]

Another application is discovery of ODBC data sources, and database servers. Rendezvous can also simply security and administration of data sources accessible by either one of these standards data access mechanisms. It can also apply to XML databases and data sources exposed by XML Databases.

• If you assume ad-hoc networking, you have to automatically define levels of access. I've always thought that the old Unix ugo (user, group, other) three-level permission system was simple and elegant, and if you replace the somewhat arbitrary "group" with "on my buddy list", you get something quite powerful. Which leads me to...

• Buddy lists ought to be supported as a standard feature of many apps, and in a consistent way. What's more, our address books really ought to make it easy to indicate who is in a "buddy list" and support numerous overlapping lists for different purposes.
  
• Every application ought to expose some version of its data as an XML feed via some well-defined and standard access mechanism. It strikes me that one of the really big wins that fueled the early web was a simple naming scheme: you could go to a site called www.foo.com, and you'd find a web server there. While it wasn't required, it made web addresses eminently guessable. We missed the opportunity for xml.foo.com to mean "this is where you get the data feed" but it's probably still possible to come up with a simple, consistent naming scheme. And of course, if we can do it for web sites, we also need to think about how to do it for local applications, since...

The very point I continue to make about Internet Points of Presence beingactual data acces points, in short these end points should be served by database serverprocesses. This is the very basis of Virtuoso, the inevitability of this realization remains the undepinings of this product. There are other products out there that have some sense of this vision too, but there is a little snag (at least so far in my research efforts), and that is the tendency to create dedicated independent server per protocol (an ultimate integration, administration, and maintenance nightmare).

• We ought to be able to have the expectation that all applications, whether local or remote (web) will be set up for two-way interactions. That is, they can be either a source or sink of online data. So, for example, the natural complement to amazon's web services data feeds is data input (for example, the ability to comment on a book on your local blog, and syndicate the review via RSS to amazon's detail page for the book.) And that leads to:

• We really need to understand who owns what, and come up with mechanisms that protect the legitimate rights of individuals and businesses to their own data, while creating the "liquidity" and free movement of data that will fuel the next great revolution in computer functionality. (I'm doing a panel on this subject at next week's Open Source Convention, entitled "We Need a Bill of Rights for Web Services.")

• We need easy gateways between different application domains. I was recently in Finland at a Nokia retreat, and we used camera-enabled cell phones to create a mobile photoblog. That was great. But even more exciting was the ease with which I could send a photo from the phone not just to another phone but also to an email address. This is the functionality that enabled the blog gateway, but it also made it trivial to send photos home to my family and friends. Similarly, I often blog things that I hear on mailing lists, and read many web sites via screen-scraping enabled email lists. It would be nice to have cross-application gateways be a routine part of software, rather than something that has to be hacked on after the fact.

This Blog Site is actually powered by Virtuoso 3.2 (has been doing so prior to the announcement). Hmm. product utilization preceding press release? Why not?

OpenLink adds Weblog client and server functionality to
Virtual Database Engine for SQL, XML, and Web Services

Burlington, MA. June 25, 2003 - OpenLink Software, Inc., a leading provider of universal data access and enterprise information integration middleware, announces Virtuoso 3.2 — the latest edition of its cross platform Virtual Database for SQL, XML, and Web Services — for Mac® OS X.

The new release incorporates full client and server support for the Blogger, Moveable Type, and MetaWeblog APIs, providing users with choice over location, format, data storage, development environment, and host operating system, for personal, community, and corporate Weblogs. The new release also facilitates the transparent integration of Weblog data with other enterprise data sources.

Full Press Release

Putting together the community site took 5 minutes and it basically involved the following steps:

1. Standard installation from installer program (Mac OS X in this case, but Windows, Linux, and UNIX supported)

2. Creation of WebDAV user account for WebDAV repository (where all the gems reside)

3. Clicking on the "Generate Web Site" button situated in the Weblog menu tree with the Virtuoso HTML based Admin UI

4. Filled up my channel and blogrolls by asking Virtuoso to use it’s very old web content aggregation functionality

5. Setup my upstreams (so that I post once and propagate to my numerous blog sites on a conditional basis)

6. Create a Virtuoso HTTP Virtual Domain for the community/personal Blog

7. Start blogging using any Blog Client that supports; Blogger API, MetaWeblog, or Moveable Type

No more no less. Most importantly I have a choice of programming languages (VSP, VSX, PHP, ASP.NET, JSP, Perl, Python), operating systems, and databases that constitute the shape and form of my blog home.

See the Virtuoso FAQ for how this all comes together.

1. Migrate my Radio Blog Web to a Virtuoso Blog Server (it is a  Web Log server that supports; Blogger API 1.0/2.0, Meta-Weblog API, Moveable Type, and xmlStorageSystem)
2. Continue to use Radio as my desktop blogging tool, but also as the local blog server gateway for other tools that I use such as w.bloggar, FM Radio and Newzcrawler

How was this acheived?

1. I had to reconfigure the Radio #upstream.xml file so that it points to my Virtuoso Server for xmlStorageSystem Web Publishing

• This is my modified version of #upstream.xml
  <!-- edited with XMLSPY v5 rel. 3 U (http://www.xmlspy.com) by Kingsley Idehen (OpenLink Software) -->
  <upstream type="xmlStorageSystem" version="1.0">
    <!--This is my Virtuoso WebDAV account-->
    <usernum>kingsley</usernum>
    <name>Kingsley Idehen</name>
    <!--This is my Radio Password Name Reference-->
    <passwordName>default</passwordName>
    <!--This is the Virtuoso instance reference-->
    <server>demo2.usnet.private</server>
    <!--Virtuoso HTTP Server Instance Port Number-->
    <port>8890</port>
    <protocol>soap</protocol>
    <!--Virtuoso XML-RPC or SOAP Endpoint-->
    <rpcPath>/xmlStorageSystem</rpcPath>
    <soapAction>/xmlStorageSystem</soapAction>
  </upstream>

• You also have to make the following change via the following Userland Radio menu path "Radio"->Window->Radio.root->user->radio->prefs->upstream->servers:
  'serverCapabilities'->flError = true;

2. Publish my local Radio site, this time to Virtuoso rather than the Userland Community Server destination

New Architecture

----------------------
| Blogging Clients
---------------------
      |
------------
| Local Radio Userland Web Server
--------------------------------
      |
-----------
| Virtuoso Server (RSS, RDF, XML, SQL etc.. in one place for further use)
----------------------

End result is productive blogging, and reusable content storage in my Virtuoso knowledgebase.