Details
Kingsley Uyi Idehen
Lexington, United States
Subscribe
Post Categories
Subscribe
Recent Articles
Display Settings
|
Virtuoso Chronicles from the Field: Nepomuk, KDE, and the quest for a sophisticated RDF DBMS.
For this particular user experience chronicle, I've simply
inserted the content of Sebastian Trueg's post titled: What We Did Last Summer (And the Rest of 2009)
– A Look Back Onto the Nepomuk Development Year ..., directly
into this post, without any additional commentary or
modification.
2009 is over. Yeah, sure, trueg, we know that, it has been
over for a while now! Ok, ok, I am a bit late, but still I
would like to get this one out - if only for my archive. So here
goes.
Let’s start with the major topic of 2009 (and also the beginning
of 2010): The new Nepomuk database backend: Virtuoso. Everybody who used Nepomuk had the
same problems: you either used the sesame2 backend which depends on Java and
steals all of your memory or you were stuck with Redland which had the
worst performance and missed some SPARQL features making important parts of
Nepomuk like queries unusable. So more than a year ago I had
the idea to use the one GPL’ed database server out there that
supported RDF in a professional manner: OpenLink’s
Virtuoso. It has all the features we need,
has a very good performance, and scales up to dimensions we will
probably never reach on the desktop (yeah, right, and 64k main
memory will be enough forever!). So very early I started
coding the necessary Soprano plugin which would talk to a locally
running Virtuoso server through ODBC. But since I ran into tons of small
problems (as always) and got sidetracked by other tasks I did not
finish it right away. OpenLink, however, was very interested in the
idea of their server being part of every KDE installation (why
wouldn’t they ;)). So they not only introduced a lite-mode which makes Virtuoso suitable
for the desktop but also helped in debugging all the problems that
I had left. Many test runs, patches, and a Virtuoso 5.0.12 release
later I could finally announce the Virtuoso
integration as usable.
Then end of last year I dropped the support for sesame2 and
redland. Virtuoso is now the only supported database backend. The
reason is simple: Virtuoso is way more powerful than the rest - not
only in terms of performance - and it is fully implemented in
C(++) without any traces of Java. Maybe even
more important is the integration of the full text index which
makes the previously used CLucene index unnecessary. Thus, we can
finally combine full text and graph queries in one SPARQL query.
This results in a cleaner API and way faster return of search
results since there is no need to combine the results from several
queries anymore. A direct result of that is the new Nepomuk Query API which I will discuss
later.
So now the only thing I am waiting for is the first bugfix
release of Virtuoso 6, i.e. 6.0.1 which will fix the bugs that make
6.0.0 fail with Nepomuk. Should be out any day now. :)
The Nepomuk Query API
Querying data in
Nepomuk pre-KDE-4.4 could be done in one of two ways: 1. Use the
very limited capabilities of the ResourceManager to list resources with
certain properties or of a certain type; or 2. Write your own
SPARQL query using ugly QString::arg
replacements.
With the introduction of Virtuoso and its awesome power we can
now do pretty much everything in one query. This allowed me to finally create a query API for KDE:
Nepomuk::Query::Query and friends. I won’t
go into much detail here since I did that before.
All in all you should remember one thing: whenever you think
about writing your own SPARQL query in a KDE application - have a
look at libnepomukquery. It is very likely that you can avoid the
hassle of debugging a query by using the query API.
The first nice effect of the new API (apart from me using it all
over the place obviously) is the new query interface in Dolphin.
Internally it simply combines a bunch of Nepomuk::Query::Term objects into a
Nepomuk::Query::AndTerm. All very readable
and no ugly query strings.
Dolphin Search Panel in KDE SC 4.4
Shared Desktop Ontologies
An important part of the Nepomuk
research project was the creation of a set of ontologies for describing desktop resources
and their metadata. After the Xesam
project under the umbrella of freedesktop.org had been convinced to use
RDF for describing file metadata they developed their own ontology.
Thanks to Evgeny (phreedom) Egorochkin and Antonie Mylka both the
Xesam ontology and the Nepomuk Information Elements Ontology were already
very close in design. Thus, it was relatively easy to merge the two
and be left with only one ontology to support. Since then not only
KDE but also Strigi and Tracker are using the Nepomuk ontologies.
At the Gran Canaria Desktop Summit I met some of the guys from
Tracker and we tried to come up with a plan to create a joint
project to maintain the ontologies. This got off to a rough start
as nobody really felt responsible. So I simply took the initiative
and released the shared-desktop-ontologies version 0.1 in
November 2009. The result was a s***-load of hate-mails and bug
reports due to me breaking KDE build. But in the end it was worth
it. Now the package is established and other projects can start to
pick it up to create data compatible to the Nepomuk system and
Tracker.
Today the ontologies (and the shared-desktop-ontologies package)
are maintained in the Oscaf project at Sourceforge. The situation
is far from perfect but it is a good start. If you need specific
properties in the ontologies or are thinking about creating one for
your own application - come and join us in the bug tracker…
Timeline KIO Slave
It was at the Akonadi meeting that Will Stephenson and myself
got into talking about mimicking some Zeitgeist functionality through Nepomuk.
Basically it meant gathering some data when opening and when saving
files. We quickly came up with a hacky patch for KIO and KFileDialog which covered most cases and
allowed us to track when a file was modified and by which
application. This little experiment did not leave that state though
(it will, however, this year) but another one did: Zeitgeist also
provides a fuse filesystem which allows to browse the files by
modification dates. Well, whatever fuse can do, KIO can do as well.
Introducing the timeline:/ KIO slave which
gives a calendar view onto your files.
Tips And Tricks
Well, I thought I would mention the Tips And Tricks section I wrote for the
techbase. It might not be a big deal but I
think it contains some valuable information in case you are using
Nepomuk as a developer.
Google Summer Of Code 2009
This time around I had the privilege to mentor two students in the Google Summer
of Code. Alessandro Sivieri and Adam Kidder did outstanding work on
Improved Virtual Folders and the Smart File Dialog.
Adam’s work lead me to some heavy improvements in the Nepomuk
KIO slaves myself which I only finished this week (more details on
that coming up). Alessandro continued his work on faceted file
browsing in KDE and created:
Sembrowser
Alessandro is following up on his work to make faceted file
browsing a reality in 2010 (and KDE SC 4.5). Since it was too late
to get faceted browsing into KDE SC 4.4 he is working on Sembrowser, a stand-alone faceted file
browser which will be the grounds for experiments until the code is
merged into Dolphin.
Faceted Browsing in KDE with
Sembrowser
Nepomuk Workshops
In 2009 I organized the first Nepomuk workshop in Freiburg,
Germany. And also the second one. While I reported properly on the first one I
still owe a summary for the second one. I will get around to that -
sooner or later. ;)
CMake Magic
Soprano gives us a nice command line tool to
create a C++ namespace from an ontology file: onto2vocabularyclass. It produces nice
convenience namespaces like Soprano::Vocabulary::NAO. Nepomuk adds
another tool named nepomuk-rcgen. Both were a bit clumsy to
use before. Now we have nice cmake macros which make it very simple
to use both.
See the techbase article on how to use the new
macros.
Bangarang
Without my knowledge (imagine that!) Andrew Lake created
an amazing new media player named Bangarang - a Jamaican word for noise,
chaos or disorder. This player is Nepomuk-enabled in the sense
that it has a media library which lets you browse your media files
based on the Nepomuk data. It remembers the number of times a song
or a video has been played and when it was played last. It allows
to add detail such as the TV series name, season, episode number,
or actors that are in the video - all through Nepomuk (I hope we
will soon get tvdb integration).
Edit metadata directly in Bangarang
Dolphin showing TV episode metadata
created by Bangarang
And of course searching for it works,
too...
And it is pretty, too...
I am especially excited about this since finally applications
not written or mentored by me start contributing Nepomuk data.
Gran Canaria Desktop Summit
2009 was also the year of the first Gnome-KDE joint-conference.
Let me make a bulletin for completeness and refer to my previous blog post reporting on my
experiences on the island.
Well, that was by far not all I did in 2009 but I think I
covered most of the important topics. And after all it is ‘just a
blog entry’ - there is no need for completeness. Thanks for
reading.
"
01/28/2010 11:14 GMT-0500 |
Modified: 02/01/2010 09:02
GMT-0500 |
New ADO.NET 3.x Provider for Virtuoso Released (Update 2)
I am pleased to announce the immediate availability of the
Virtuoso ADO.NET 3.5 data provider for
Microsoft's .NET platform.
What is it?
A data access driver/provider that provides conceptual entity oriented access to RDBMS data managed by Virtuoso. Naturally,
it also uses Virtuoso's in-built virtual / federated database layer to provide access to
ODBC and JDBC accessible RDBMS engines such as:
Oracle (7.x to latest), SQL
Server (4.2 to latest), Sybase, IBM Informix (5.x to latest), IBM DB2,
Ingres (6.x to latest), Progress (7.x to
OpenEdge), MySQL, PostgreSQL, Firebird, and others using our ODBC or JDBC
bridge drivers.
Benefits?
Technical:
It delivers an Entity-Attribute-Value + Classes &
Relationships model over disparate data sources that are
materialized as .NET Entity Framework Objects, which are then
consumable via ADO.NET Data Object Services, LINQ for Entities, and
other ADO.NET data consumers.
The provider is fully integrated into Visual Studio 2008 and
delivers the same "ease of use" offered by Microsoft's own SQL
Server provider, but across Virtuoso, Oracle, Sybase, DB2,
Informix, Ingres, Progress (OpenEdge), MySQL, PostgreSQL,
Firebird, and others. The same benefits also apply uniformly to
Entity Frameworks compatibility.
Bearing in mind that Virtuoso is a multi-model (hybrid) data
manager, this also implies that you can use .NET Entity Frameworks
against all data managed by Virtuoso. Remember, Virtuoso's SQL
channel is a conduit to Virtuoso's core; thus, RDF (courtesy of
SPASQL as already implemented re. Jena/Sesame/Redland providers), XML, and other data
forms stored in Virtuoso also become accessible via .NET's Entity
Frameworks.
Strategic:
You can choose which entity oriented data access model works
best for you: RDF Linked Data & SPARQL or .NET Entity Frameworks &
Entity SQL. Either way, Virtuoso delivers
a commercial grade, high-performance, secure, and scalable
solution.
How do I use it?
Simply follow one of guides below:
Note: When working with external or 3rd party databases,
simply use the Virtuoso Conductor to link the external data source
into Virtuoso. Once linked, the remote tables will simply be
treated as though they are native Virtuoso tables leaving the
virtual database engine to handle the rest.
This is similar to the role the Microsoft JET engine played in the
early days of ODBC, so if you've ever linked an ODBC data source
into Microsoft Access, you are ready to do the same using
Virtuoso.
Related
01/08/2009 04:36 GMT-0500 |
Modified: 01/08/2009 09:12
GMT-0500 |
Virtuoso Universal Server 5.0.4 Release Details
We've just released version 5.0.4 of the Virtuoso Universal
Server platform for SQL, XML, and RDF. The new release includes the
following enhancements:
Web Server:
- HTTP 1.1 compliant Transparent
content-negotiation in URL-rewrite rules for Linked Data
Deployment.
RDF Data Management:
- New providers for the Jena, Sesame
and Redland frameworks
- support for SPARQL INSERT and
UPDATE via HTTP POST
- New SPARQL-BI extenstions that make
Business Intelligence feasible via SPARQL
- new "rdf_sink" folder for handling
HTTP PUTs into WebDAV that automatically sync with Quad
Store.
- There are new Sponger (RDFizer)
cartridges that map Amazon book-search results to the Biliographic
Ontology, supports production of Linked Data from OAI, XBRL, and
Yahoo finance data sources.
- HTTPS protocol support added to
Sponger
- performance optimizations for
SPARQL `DESCRIBE' and `CONSTRUCT', alongside general performance
enhancements for RDF data set loading.
Core DBMS Engine:
- PHP hosting a module re-implemented
as a Virtuoso plugin inline with otherlanguage hosting
modules
- improved deadlock condtion
management
- enhanced POP and FTP server side
protocol implementations that allow larger data transfers.
Additional Information
02/04/2008 14:25 GMT-0500 |
Modified: 02/04/2008 20:30
GMT-0500 |
W3C's SPARQLing Data Access Ingenuity
The W3C officially unveiled the SPARQL Query Language today via
a press release titled: W3C Opens Data on the Web with SPARQL.
A query language for the burgeoning Structured & Linked Data Web (aka Semantic Web / Giant Global Graph). Like SQL, for
the Relational Data Model, it provides a query language for the
Graph based RDF Data Model.
It's also a REST or SOAP
based Web Service that exposes SPARQL access to RDF Data via an
endpoint.
In addition, it's also a Query Results Serialization format that
includes XML and JSON
support.
Why is it Important?
It brings important clarity to the notion of the "Web as a
Database" by transforming existing Web Sites, Portals, and Web
Services into bona fide corpus of Mesh-able (rather than Mash-able)
Data Sources. For instance, you can perform queries that join one
or more of the aforementioned data sources in exactly the same
manner (albeit different syntax) as you would one or more SQL
Tables.
Example:
-- SPARQL equivalent of SQL SELECT * against my personal data
space hosted FOAF file
SELECT DISTINCT ?s ?p ?o
FROM <http://myopenlink.net/dataspace/person/kidehen>
WHERE {?s ?p ?o}
-- SPARQL against my social network -- Note: My SPARQL will be
beamed across all of contacts in the social networks of my contacts
as long as they are all HTTP URI based within each data space
PREFIX foaf: <http://xmlns.com/foaf/0.1/>
SELECT DISTINCT ?Person
FROM <http://myopenlink.net/dataspace/person/kidehen>
WHERE {?s a foaf:Person; foaf:knows ?Person}
Note: you can use the basic SPARQL
Endpoint, SPARQL Query By Example, or SPARQL Query Builder Demo tool to experiment
with the demonstration queries above.
How Do I use It?
SPARQL is implemented by RDF Data Management Systems (Triple or
Quad Stores) just as SQL is implemented by Relational Database
Management Systems. The aforementioned data management systems will
typically expose SPARQL access via a SPARQL endpoint.
Where are it's implementations?
A SPARQL implementors Testimonial page accompanies the SPARQL
press release. In addition the is a growing collection of
implementations on the ESW Wiki Page for SPARQL compliant RDF Triple
& Quad Stores.
Is this really a big deal?
Yes! SPARQL facilitates an unobtrusive manifestation of a Linked Data
Web by way of natural extension of the existing Document Web
i.e these Web enclaves co-exist in symbiotic fashion.
As DBpedia
very clearly demonstrates, Linked Data makes the Semantic Web
demonstrable and much easier to comprehend. Without SPARQL there
would be no mechanism for Linked Data deployment, and without Linked
Data there is no mechanism for Beaming Queries (directly or
indirectly) across the Giant Global Graph of data hosted by Social
Networks, Shard Bookmarks Services, Weblogs, Wikis, RSS/Atom/OPML
feeds, Photo Galleries and other Web accessible Data Sources (Data
Spaces).
Related items
Detailed
SPARQL Query Examples using SIOC Data Spaces
01/15/2008 22:58 GMT-0500 |
Modified: 01/17/2008 15:41
GMT-0500 |
OpenLink Ajax Toolkit (OAT) 2.6 Released!
OpenLink Software are
pleased to announce release 2.6 of the OpenLink AJAX Toolkit (OAT).
New Semantic Data Web related features and enhancements
include:
* A Javascript-based Fresnel processor
enabling declarative RDF-based display templates for RDF Data
Sources
* An XSLT template for generating
HTML pages from the Fresnel processor's XML output
* Enhanced Javascript-based N3/Turtle
parser
Related Items:
08/01/2007 18:34 GMT-0500 |
Modified: 08/01/2007 14:49
GMT-0500 |
Enterprise 0.0, Linked Data, and Semantic Data Web
Last week we officially released
Virtuoso 5.0.1 (in Commercial and Open Source Editions). The
press release provided us with an official mechanism and timestamp
for the current Virtuoso feature set.
A vital component of the new Virtuoso release is the
finalization of our SQL to RDF mapping functionality -- enabling
the declarative mapping of SQL Data to RDF. Additional technical
insight covering other new features (delivered and pending) is
provided by Orri Erling, as
part of a series of post-Banff posts.
Why is SQL to RDF Mapping a Big Deal?
A majority of the world's data (especially in the enterprise
realm) resides in SQL Databases. In addition, Open Access to the
data residing in said databases remains the biggest challenge to
enterprises for the following reasons:
- SQL Data Sources are inherently heterogeneous because they are
acquired with business applications that are in many cases
inextricably bound to a particular DBMS engine
- Data is predictably dirty
- DBMS vendors ultimately hold the data captive and have
traditionally resisted data access standards such as ODBC (*trust
me they have, just look at the unprecedented bad press associated
with ODBC the only truly platform independent data access API. Then
look at how this bad press arose..*)
Enterprises have known from the beginning of modern corporate
times that data access, discovery, and manipulation capabilities
are inextricably linked to the "Real-time Enterprise" nirvana
(hence my use of 0.0 before this becomes 3.0).
In my experience, as someone whose operated in the data access
and data integration realms since the late '80s, I've painfully
observed enterprises pursue, but unsuccessfully attain, full
control over enterprise data (the prized asset of any organization)
such that data-, information-, knowledge-workers are just a click
away from commencing coherent platform and database independent
data drill-downs and/or discovery that transcend intranet,
internet, and extranet boundaries -- serendipitous interaction with
relevant data, without compromise!
Okay, situation analysis done, we move on..
At our most recent (12th
June) monthly Semantic
Web Gathering, I unveiled to TimBL and a host
of other attendees a simple, but powerful, demonstration of how
Linked Data,
as an aspect of the
Semantic Data Web, can be applied to enterprise data
integration challenges.
Actual SQL to RDF Mapping Demo / Experiment
Hypothesis
A SQL Schema can be effectively mapped declaratively to RDF such
that SQL Rows morph into RDF Instance Data (Entity Sets) based on
the Concepts & Properties defined in a Concrete Conceptual Data
Model oriented Data Dictionary ( RDF Schema and/or
OWL
Ontology). In addition, the solution must demonstrate how
"Linked Data in the Web" is completely different from "Data on the
Web" or "Linked Data on the Web" (btw - Tom Heath eloquently
unleashed this point in his recent
podcast interview with Talis).
Apparatus
An Ontology - in this case we simply derived the
Northwind Ontology from the XML Schema based CSDL ( Conceptual
Schema Definition Language) used by Microsoft's public Astoria demo
(specifically the
Northwind Data Services demo). SQL Database Schema -
Northwind (comes bundled with ACCESS, SQL Server, and Virtuoso)
comprised of tables such as: Customer,
Employee,
Product,
Category,
Supplier,
Shipper
etc. OpenLink
Virtuoso - SQL DBMS Engine (although this could have been any
ODBC
or JDBC
accessible Database),
SQL-RDF Metaschema Language, HTTP URL-rewriter, WebDAV Engine,
and DBMS hosted XSLT processor Client Tools - iSPARQL Query Builder,
RDF
Browser (which could also have been Tabulator or DISCO
or a standard Web Browser)
Experiment / Demo
- Declaratively map the Northwind SQL Schema to RDF using the
Virtuoso Meta Schema Language (see:
Virtuoso PL based Northwind_SQL_RDF script)
- Start browsing the data by clicking on the URIs that represent
the RDF Data Model Entities resulting from the SQL to RDF
Mapping
Observations
- Via a single Data Link click I was able to obtain specific
information about the Customer represented by the URI "ALFKI"
(act of URI Dereferencing as you would an Object ID in an Object or
Object-Relational Database)
- Via a
Dynamic Data Page I was able to explore all the entity
relationships or specific entity data (i.e Exploratory or Entity
specific dereferencing) in the Northwind Data Space
- I was able to perform similar exploration (as per item 2) using
our
OpenLink Browser.
Conclusions
The vision of data, information, or knowledge at your fingertips
is nigh! Thanks to the infrastructure provided by the Semantic Data
Web (URIs, RDF
Data Model, variety of RDF Serialization Formats[1][2][3],
and Shared Data Dictionaries / Schemas / Ontologies [1][2][3][4][5])
it's now possible to Virtualize enterprise data from the Physical
Storage Level, through the Logical Data Management Levels
(Relational), up to a Concrete Conceptual Model (Graph) without
operating system, development environment or framework, or database
engine lock-in.
Next Steps
We produce a shared ontology for the CRM and Business Reporting
Domains. I hope this experiment clarifies how this is quite
achievable by converting XML Schemas to RDF Data Dictionaries (RDF
Schemas or Ontologies). Stay tuned :-)
Also watch TimBL amplify and
articulate Linked Data value in a recent interview.
Other Related Matters
To deliver a mechanism that facilitates the crystallization of
this reality is a contribution of boundless magnitude (as we shall
all see in due course). Thus, it is easy to understand why even
"her majesty", the queen of England, simply had to get in on the
act and appoint TimBL to the
"British Order of Merit" :-)
Note: All of the demos above now work with IE & Safari (a
"remember what Virtuoso is epiphany") by simply putting Virtuoso's
DBMS hosted XSLT engine to use :-) This also applies to my earlier
collection of demos from the
Hello Data Web and other
Data Web & Linked Data related demo style posts.
06/14/2007 15:28 GMT-0500 |
Modified: 02/04/2008 23:19
GMT-0500 |
"Free" Databases: Express vs. Open-Source RDBMSs
Very detailed and insightful peek into the state of affairs re.
database engines (Open & Closed Source).
I added the missing piece regarding the "Virtuoso Conductor"
(the Web based Admin UI for Virtuoso) to the original post below. I
also added a link to our live SPARQL Demo so that anyone interested
can start playing around with SPARQL and SPARQL integrated into SQL
right away.
Another good thing about this post is the vast amount of
valuable links that it contains. To really appreciate this point
simply visit my Linkblog (excuse the current layout :-) - a Tab if
you come in via the front door of this Data Space (what
I used to call My Weblog Home
Page).
"Free" Databases: Express vs. Open-Source RDBMSs: "Open-source relational database management
systems (RDBMSs) are gaining IT mindshare at a rapid pace. As an
example, BusinessWeek's February 6, 2006 '
Taking On the Database
Giants ' article
asks 'Can open-source upstarts compete with Oracle, IBM, and
Microsoft?' and then provides the answer: 'It's an uphill battle,
but customers are starting to look at the
alternatives.'
There's no shortage of
open-source alternatives to look at. The BusinessWeek
article concentrates on MySQL,
which BW says 'is trying to be the Ikea of the database
world: cheap, needs some assembly, but has a sleek, modern design
and does the job.' The article also discusses Postgre[SQL] and Ingres,
as well as EnterpriseDB,
an Oracle clone created from PostgreSQL code*. Sun includes
PostgreSQL
with Solaris 10 and, as of April 6, 2006, with Solaris
Express.**
*Frank Batten, Jr., the investor who originally
funded Red Hat, invested a reported $16 million into Great Bridge
with the hope of making a business
out of providing paid support to PostgreSQL users. Great Bridge stayed in business only 18
months , having
missed an opportunity to sell the business to Red
Hat and finding that
selling $50,000-per-year support packages
for an open-source database wasn't
easy. As Batten concluded, 'We could not get customers to pay us
big dollars for support contracts.' Perhaps EnterpriseDB will be
more successful with a choice of
$5,000, $3,000, or $1,000 annual support subscriptions
.
**Interestingly,
Oracle announced in November 2005 that Solaris 10 is 'its
preferred development and deployment platform for most x64
architectures, including x64 (x86, 64-bit) AMD Opteron and Intel
Xeon processor-based systems and Sun's UltraSPARC(R)-based
systems.'
There is a surfeit of reviews
of current MySQL, PostgreSQL and—to a lesser extent—Ingres
implementations. These three open-source RDBMSs come with their own
or third-party management tools. These systems compete against free
versions of commercial (proprietary) databases: SQL Server 2005
Express Edition (and its MSDE 2000 and 1.0 predecessors),
Oracle Database 10g Express Edition, IBM DB2 Express-C, and Sybase ASE
Express Edition for Linux where database size and processor
count limitations aren't important. Click
here for a summary of recent InfoWorld reviews of the
full versions of these four databases plus MySQL, which should be
valid for Express editions also. The FTPOnline Special
Report article, 'Microsoft SQL Server Turns 17,' that contains
the preceding table is here
(requires registration.)
SQL Server 2005
Express Edition SP-1 Advanced Features
SQL Server 2005 Express Edition with Advanced Features enhances
SQL Server 2005 Express Edition (SQL Express or SSX) dramatically,
so it deserves special treatment here. SQL Express gains full text
indexing and now supports SQL Server Reporting Services (SSRS) on
the local SSX instance. The SP-1 with Advanced Features setup
package, which Microsoft released on April 18, 2006, installs the
release version of SQL Server Management Studio Express (SSMSE) and
the full version of Business Intelligence Development Studio (BIDS)
for designing and editing SSRS reports. My 'Install
SP-1 for SQL Server 2005 and Express' article for FTPOnline's
SQL Server
Special Report provides detailed, illustrated installation
instructions for and related information about the release version
of SP-1. SP-1 makes SSX the most capable of all currently available
Express editions of commercial RDBMSs for Windows.
OpenLink Software's
Virtuoso Open-Source Edition
OpenLink Software announced an
open-source
version of it's Virtuoso Universal Server
commercial DBMS on April 11, 2006. On the initial date of this
post, May 2, 2006, Virtuoso Open-Source Edition (VOS) was virtually
under the radar as an open-source product. According to this press
release, the new edition includes:
-
SPARQL
compliant RDF Triple Store
- SQL-200n Object-Relational Database Engine (SQL, XML, and Free
Text)
- Integrated BPEL Server and Enterprise Service Bus
- WebDAV and Native File Server
- Web Application Server that supports PHP, Perl, Python,
ASP.NET, JSP, etc.
- Runtime Hosting for Microsoft .NET, Mono, and Java
VOS only lacks the virtual server and replication features that are
offered by the commercial edition. VOS includes a Web-based
administration tool called the "Virtuoso Conductor" According to
Kingsley Idehen's Weblog, 'The Virtuoso build scripts have been
successfully tested on Mac OS X (Universal Binary Target), Linux,
FreeBSD, and Solaris (AIX, HP-UX, and True64 UNIX will follow
soon). A Windows Visual Studio project file is also in the works
(ETA some time this week).'
InfoWorld's Jon Udell has tracked Virtuoso's progress
since
2002, with an additional
article in 2003 and a one-hour
podcast with Kingsley Idehen on April 26, 2006. A major talking
point for Virtuoso is its support for Atom 0.3 syndication and
publication, Atom 1.0 syndication and (forthcoming) publication,
and future support for Google's GData
protocol, as mentioned in
this Idehen post. Yahoo!'s Jeremy
Zawodny points out that the 'fingerprints' of
Adam Bosworth, Google's VP of Engineering and the primary force
behind the development of Microsoft Access, 'are all over GData.'
Click
here to display a list of all OakLeaf posts that mention Adam
Bosworth.
One application for the GData protocol is querying and updating the
Google Base database independently of the Google Web client, as
mentioned by Jeremy: 'It's not about building an easier onramp to
Google Base. ... Well, it is. But, again, that's the small stuff.'
Click
here for a list of posts about my experiences with Google Base.
Watch for a future OakLeaf post on the subject as the GData APIs
gain ground.
Open-Source and Free
Embedded Database Contenders
Open-source and free embedded
SQL databases are gaining importance as the number and types of
mobile devices and OSs proliferate. Embedded databases usually
consist of Java classes or Windows DLLs that are designed to
minimize file size and memory consumption. Embedded databases avoid
the installation hassles, heavy resource usage and maintenance cost
associated with client/server RDBMSs that run as an operating
system service.
Andrew Hudson's December 2005
'Open Source
databases rounded up and rodeoed' review for The Enquirer
provides brief descriptions of one commercial and eight open source
database purveyors/products: Sleepycat, MySQL, PostgreSQL, Ingres,
InnoBase, Firebird, IBM Cloudscape (a.k.a, Derby), Genezzo, and
Oracle. Oracle Sleepycat*
isn't an SQL Database, Oracle InnoDB* is an OEM database
engine that's used by MySQL, and Genezzo is a multi-user, multi-server
distributed database engine written in Perl. These special-purpose
databases are beyond the scope of this post.
*
Oracle purchased Sleepycat
Software, Inc. in February 2006 and purchased Innobase OY in October 2005
. The press release states: 'Oracle
intends to continue developing the InnoDB technology and expand our
commitment to open source software.'
Derby
is an
open-source release by the Apache
Software Foundation of the Cloudscape
Java-based database that IBM acquired when it bought Informix
in 2001. IBM offers a commercial release of Derby as
IBM Cloudscape 10.1. Derby is a Java class library that has a
relatively light footprint (2 MB), which make it suitable for
client/server synchronization with the IBM DB2 Everyplace Sync
Server in mobile
applications. The IBM DB2 Everyplace Express Edition isn't open
source or free*, so it doesn't qualify for this post. The same is
true for the corresponding Sybase SQL Anywhere
components.**
* IBM DB2
Everyplace Express Edition with synchronization costs $379 per
server (up to two processors) and $79 per user. DB2 Everyplace
Database Edition (without DB2 synchronization) is $49 per user.
(Prices are based on those when
IBM announced version 8 in November
2003.)
** Sybase's
iAnywhere subsidiary calls SQL Anywhere 'the industry's leading
mobile database.' A Sybase SQL Anywhere Personal DB seat license
with synchronization to SQL Anywhere Server is $119; the cost
without synchronization wasn't available from the Sybase Web site.
Sybase SQL Anywhere and IBM DB2 Everyplace perform similar
replication functions.
Sun's Java
DB, another commercial version of Derby, comes with
the Solaris
Enterprise Edition, which bundles Solaris 10, the Java
Enterprise System, developer tools, desktop infrastructure and N1
management software. A recent Between the Lines blog entry by
ZDNet's David Berlind waxes enthusiastic over the use of Java DB embedded in a
browser to provide offline persistence. RedMonk analyst
James
Governor and eWeek's Lisa Vaas
wrote about the use of Java DB as a local data store when Tim Bray announced
Sun's Derby derivative and
Francois Orsini demonstrated Java DB embedded in the Firefox
browser at the ApacheCon 2005 conference.
Firebird
is
derived from Borland's InterBase 6.0 code, the first commercial
relational database management system (RDBMS) to be released as
open source. Firebird has excellent support for SQL-92 and comes in
three versions: Classic, SuperServer and Embedded for Windows,
Linux, Solaris, HP-UX, FreeBSD and MacOS X. The embedded version
has a 1.4-MB footprint. Release Candidate 1 for Firebird 2.0 became
available on March 30, 2006 and is a major improvement over earlier
versions. Borland
continues to promote InterBase, now at version 7.5, as a
small-footprint, embedded database with commercial Server and
Client licenses.
SQLite
is a
featherweight C library for an embedded database that implements
most SQL-92 entry- and transitional-level requirements (some
through the JDBC driver) and supports transactions within a tiny
250-KB code footprint. Wrappers
support a multitude of languages and operating systems, including
Windows CE, SmartPhone, Windows Mobile, and Win32. SQLite's primary
SQL-92 limitations
are lack of nested transactions, inability to alter a table design
once committed (other than with RENAME TABLE and ADD COLUMN
operations), and foreign-key constraints. SQLite provides read-only
views, triggers, and 256-bit encryption of database files. A
downside is the the entire database file is locked
when while a transaction is in progress. SQLite uses file
access permissions in lieu of GRANT and REVOKE commands. Using
SQLite involves no license; its code is entirely in the public
domain.
The Mozilla
Foundation's Unified Storage
wiki says this about SQLite: 'SQLite will be the back end for
the unified store [for Firefox]. Because it implements a SQL
engine, we get querying 'for free', without having to invent our
own query language or query execution system. Its code-size
footprint is moderate (250k), but it will hopefully simplify much
existing code so that the net code-size change should be smaller.
It has exceptional performance, and supports concurrent access to
the database. Finally, it is released into the public domain,
meaning that we will have no licensing issues.'
Vieka Technology, Inc.'s
eSQL 2.11
is a port of SQLite to Windows Mobile (Pocket PC and Smartphone)
and Win32, and includes development tools for Windows devices and
PCs, as well as a .NET native data provider. A conventional ODBC
driver also is available. eSQL for Windows (Win32) is free for
personal and commercial use; eSQL for Windows Mobile requires a
license for commercial (for-profit or business) use.
HSQLDB
isn't on most
reviewers' radar, which is surprising because it's the default
database for OpenOffice.org (OOo) 2.0's
Base
suite member. HSQLDB 1.8.0.1 is an open-source (BSD license) Java
dembedded database engine based on Thomas Mueller's original
Hypersonic SQL Project. Using OOo's Base feature requires
installing the Java 2.0 Runtime Engine (which is not open-source)
or the presence of an alternative open-source engine, such as
Kaffe. My prior posts about OOo Base and HSQLDB are
here,
here and
here.
The HSQLDB 1.8.0
documentation on SourceForge states the following regarding
SQL-92 and later conformance:
HSQLDB 1.8.0 supports the dialect of SQL
defined by SQL standards 92, 99 and 2003. This means where a
feature of the standard is supported, e.g. left outer join, the
syntax is that specified by the standard text. Many features of
SQL92 and 99 up to Advanced Level are supported and here is support
for most of SQL 2003 Foundation and several optional features of
this standard. However, certain features of the Standards are not
supported so no claim is made for full support of any level of the
standards.
Other less well-known embedded databases designed
for or suited to mobile deployment are Mimer SQL Mobile and VistaDB
2.1 . Neither product is
open-source and require paid licensing; VistaDB requires a small
up-front payment by developers but offers royalty-free
distribution.
Java DB, Firebird embedded,
SQLite and eSQL 2.11 are contenders for lightweight PC and mobile
device database projects that aren't Windows-only.
SQL Server 2005
Everywhere
If you're a Windows developer,
SQL Server Mobile is the logical embedded database choice for
mobile applications for Pocket PCs and Smartphones. Microsoft's
April 19, 2006 press release delivered the news that SQL Server
2005 Mobile Editon (SQL Mobile or SSM) would gain a big brother—SQL
Server 2005 Everywhere Edition.
Currently, the SSM client is
licensed (at no charge) to run in production on devices with
Windows CE 5.0, Windows Mobile 2003 for Pocket PC or Windows Mobile
5.0, or on PCs with Windows XP Tablet Edition only. SSM also is
licensed for development purposes on PCs running Visual Studio
2005. Smart Device
replication with SQL Server 2000 SP3 and later databases has been
the most common application so far for SSM.
By the end of
2006, Microsoft will license SSE for use on all PCs
running any Win32 version or the preceding device OSs. A version of
SQL Server Management Studio Express (SSMSE)—updated to support
SSE—is expected to release by the end of the year. These features
will qualify SSE as the universal embedded database for
Windows client and smart-device applications.
For more details on SSE, read
John
Galloway's April 11, 2006 blog post and my 'SQL
Server 2005 Mobile Goes Everywhere' article for the FTPOnline Special
Report on SQL Server."
(Via OakLeaf
Systems.)
05/05/2006 16:02 GMT-0500 |
Modified: 07/21/2006 07:21
GMT-0500 |
Solutions to allow XMLHttpRequest to talk to external services
Solutions
to allow XMLHttpRequest to talk to external services: "
Over on XML.com they published
Fixing AJAX: XmlHttpRequest Considered Harmful.
This article discusses a few ways to get around the security
constraints that we have to live with in the browsers theses days,
in particular, only being able to talk to your domain via XHR.
The article walks you through three potential solutions:
-
Application proxies. Write an application in
your favorite programming language that sits on your server,
responds to
XMLHttpRequest s from users, makes the web
service call, and sends the data back to users.
-
Apache proxy. Adjust your Apache web server
configuration so that
XMLHttpRequest s can be invisibly
re-routed from your server to the target web service domain.
-
Script tag hack with application proxy
(doesn't use
XMLHttpRequest at all). Use the HTML
script tag to make a request to an application proxy
(see #1 above) that returns your data wrapped in JavaScript. This
approach is also known as On-Demand
JavaScript.
I can't wait for Trusted Relationships within the
browser - server infrastructure.
With respect to Apache proxies, these things are priceless. I
recently talked about them in relation to Migrating data
centers with zero downtime.
What do you guys think about this general issue? Have you come
up with any interesting solutions? Any ideas on how we can keep
security, yet give us the freedom that we want?
(Via Ajaxian Blog.)
Well here is what I think (actually know):
Our Virtuoso Universal
Server has been sitting waiting to deliver this for years (for
the record see the Virtuoso 2000 Press
Release). Virtuoso can proxy for disparate data sources and
expose disparate data as Well-Formed XML using an array of
vocabularies (you experience this SQL-XML integration on the fly
every time you interact with various elements of my public blog).
Virtuoso has always been able to expose Application Logic as
SOAP and/or RESTful/RESTian style XML Web Services. This
blog's search page is a simple demo of this capability.
Virtuoso is basically a Junction Box / Aggregator / Proxy for
disparate Data, Applications, Services, and BPEL compliant business
processes. AJAX clients talk to this single multi-purpose server
which basically acts as a conduit to content/data, services, and
processes (which are composite services).
BTW - there is a lot more, but for now, thou shall have to seek
in order to find :-)
11/11/2005 21:01 GMT-0500 |
Modified: 07/21/2006 07:23
GMT-0500 |
Search Beyond Google
Search
Beyond Google
Google has a large lead over its rivals in U.S. audience share,
accounting for 77 percent of all searches in August 2003 (including
searches conducted at AOL and Yahoo!, which used the Google search
engine). But in the search industry, innovation is a wild card. In
1999, you could have said that AltaVista had pretty much finished
off the search market,notes Whit Andrews, a research director at
technology advisory firm Gartner. In 1997, it was Inktomi. In 1995,
it was Yahoo!. You never know in the search business when there?s
somebody down the street who is going to make you look like
yesterday's news.
Good article on
the future of search engines. One that caught my eye was
Mooter; I'll be watching that
one closely. [via Lockergnome's Technology
News]
Great piece! I took a quick look at Mooter, and I was very
impressed. I certainly remember the internal emails that brought
AltaVista, AllTheWeb, and eventually Google to my attention. Of
course there will be search beyond Google, that's what makes us
human (our
02/17/2004 12:59 GMT-0500 |
Modified: 06/22/2006 08:56
GMT-0500 |
|
|