OpenLink Virtuoso (Product Blog)

Developer Opportunities at OpenLink Software

Tue, 07 Aug 2012 17:21:52 GMT

If it is advanced database technology, you will get to do it with us.

We are looking for exceptional talent to implement some of the hardest stuff in the industry. This ranges from new approaches to query optimization; to parallel execution (both scale up and scale out); to elastic cloud deployments and self-managing, self-tuning, fault-tolerant databases. We are most familiar to the RDF world, but also have full SQL support, and the present work will serve both use cases equally.

We are best known in the realms of high-performance database connectivity middleware and massively-scalable Linked-Data-oriented graph-model DBMS technology.

We have the basics -- SQL and SPARQL, column store, vectored execution, cost based optimization, parallel execution (local and cluster), and so forth. In short, we have everything you would expect from a DBMS. We do transactions as well as analytics, but the greater challenges at present are on the analytics side.

You will be working with my team covering:

Adaptive query optimization -- interleaving execution and optimization, so as to always make the correct plan choices based on actual data characteristics
Self-managing cloud deployments for elastic big data -- clusters that can grow themselves and redistribute load, recover from failures, etc.
Developing and analyzing new benchmarks for RDF and graph databases
Embedding complex geospatial reasoning inside the database engine. We have the basic R-tree and the OGC geometry data types; now we need to go beyond this
Every type of SQL optimizer and execution engine trick that serves to optimize for TPC-H and DS.

What do I mean by really good? It boils down to being a smart and fast programmer. We have over the years talked to people, including many who have worked on DBMS programming, and found that they actually know next to nothing of database science. For example, they might not know what a hash join is. Or they might not know that interprocess latency is in the tens of microseconds even within one box, and that in that time one can do tens of index lookups. Or they might not know that blocking on a mutex kills.

If you do core database work, we want you to know how many CPU cache misses you will have in flight at any point of the algorithm, and how many clocks will be spent waiting for them at what points. Same for distributed execution: The only way a cluster can perform is having max messages with max payload per message in flight at all times.

These are things that can be learned. So I do not necessarily expect that you have in-depth experience of these, especially since most developer jobs are concerned with something else. You may have to unlearn the bad habit of putting interfaces where they do not belong, for example. Or to learn that if there is an interface, then it must pass as much data as possible in one go.

Talent is the key. You need to be a self-starter with a passion for technology and have competitive drive. These can be found in many guises, so we place very few limits on the rest. If you show you can learn and code fast, we don't necessarily care about academic or career histories. You can be located anywhere in the world, and you can work from home. There may be some travel but not very much.

In the context of EU FP7 projects, we are working with some of the best minds in database, including Peter Boncz of CWI and VU Amsterdam (MonetDB, VectorWise) and Thomas Neumann of Technical University of Munich (RDF3X, HYPER). This is an extra guarantee that you will be working on the most relevant problems in database, informed by the results of the very best work to date.

For more background, please see the IEEE Computer Society Bulletin of the Technical Committee on Data Engineering, Special Issue on Column Store Systems.

All articles and references therein are relevant for the job. Be sure to read the CWI work on run time optimization (ROX), cracking, and recycling. Do not miss the many papers on architecture-conscious, cache-optimized algorithms; see the VectorWise and MonetDB articles in the bulletin for extensive references.

If you are interested in an opportunity with us, we will ask you to do a little exercise in multithreaded, performance-critical coding, to be detailed in a blog post in a few days. If you have done similar work in research or industry, we can substitute the exercise with a suitable sample of this, but only if this is core database code.

There is a dual message: The challenges will be the toughest a very tough race can offer. On the other hand, I do not want to scare you away prematurely. Nobody knows this stuff, except for the handful of people who actually do core database work. So we are not limiting this call to this small crowd and will teach you on the job if you just come with an aptitude to think in algorithms and code fast. Experience has pros and cons so we do not put formal bounds on this. "Just out of high school" may be good enough, if you are otherwise exceptional. Prior work in RDF or semantic web is not a factor. Sponsorship of your M.Sc. or Ph.D. thesis, if the topic is in our line of work and implementation can be done in our environment, is a further possibility. Seasoned pros are also welcome and will know the nature of the gig from the reading list.

We are aiming to fill the position(s) between now and October.

Resumes and inquiries can be sent to Hugh Williams, hwilliams@openlinksw.com. We will contact applicants for interviews.

Virtuoso 6.2 brings New Features!

Wed, 22 Sep 2010 21:08:24 GMT

Virtuoso 6.2 introduces a major number of enhancements to areas including...

Linked Data Deployment
Linked Data Middleware
Data Virtualization
Dynamic Data Exchange & Data Replication
Security

Linked Data Deployment

Feature	Description	Benefit
Automatic Deployment	Linked Data Pages are now automatically published for every Virtuoso Data Object; users need only load their data into the RDF Quad Store.	Handcrafted URL-Rewrite Rules are no longer necessary.
HTTP Metadata Enhancements	HTTP `Link:` header is used to transfer vital metadata (e.g., relationships between a Descriptor Resource and its Subject) from HTTP Servers to User Agents.	Enables HTTP-oriented tools to work with such relationships and other metadata.
HTML Metadata Embedding	HTML resource `<head />` and `<link />` elements and their `@rel` attributes are used to transfer vital metadata (e.g., relationships between a Descriptor Resource and its Subject) from HTTP Servers to User Agents.	Enables HTML-oriented tools to work with such relationships and other metadata.
Hammer Stack Auto-Discovery Patterns	HTML resource `<head />` section and `<link />` elements, the HTTP `Link:` header, and XRD-based `"host-meta"` resources collectively provide structured metadata about Virtuoso hosts, associated Linked Data Spaces, and specific Data Items (Entities).	Enables humans and machines to easily distinguish between Descriptor Resources and their Subjects, irrespective of URI scheme.

Linked Data Middleware

Feature	Description	Benefit
New Sponger Cartridges	New cartridges (data access and transformation drivers) for Twitter, Facebook, Amazon, eBay, LinkedIn, and others.	Enable users and user agents to deal with the Sponged data spaces as though they were named graphs in a quad store, or tables in an RDBMS.
New Descriptor Pages	HTML-based descriptor pages are automatically generated.	Descriptor subjects, and the constellation of navigable attribute-and-value pairs that constitute their descriptive representation, are clearly identified.
Automatic Subject Identifier Generation	De-referenceable data object identifiers are automatically created.	Removes tedium and risk of error associated with nuance-laced manual construction of identifiers.
Support for OData, JSON, RDFa	Additional data representation and serialization formats associated with Linked Data.	Increases flexibility and interoperability.

Data Virtualization

Feature	Description	Benefit
Materialized RDF Views	RDF Views over ODBC/JDBC Data Sources can now (optionally) keep the Quad Store in sync with the RDBMS data source.	Enables high-performance Faceted Browsing while remaining sensitive to changes in the RDBMS data sources.
CSV-to-RDF Transformation	Wizard-based generation of RDF Linked Data from CSV files.	Speeds deployment of data which may only exist in CSV form as Linked Data.
Transparent Data Access Binding	SPASQL (SPARQL Query Language integrated into SQL) is usable over ODBC, JDBC, ADO.NET, OLEDB, or XMLA connections.	Enables Desktop Productivity Tools to transparently work with any blend of RDBMS and RDF data sources.

Dynamic Data Exchange & Data Replication

Feature	Description	Benefit
Quad Store to Quad Store Replication	High-fidelity graph-data replication between one or more database instances.	Enables a wide variety of deployment topologies.
Delta Engine	Automated generation of deltas at the named-graph-level, matches transactional replication offered by the Virtuoso SQL engine.	Brings RDF replication on par with SQL replication.
PubSubHubbub Support	Deep integration within Quad Store as an optional mechanism for shipping deltas.	Enables push-based data replication across a variety of topologies.

Security

Feature	Description	Benefit
WebID support at the DBMS core	Use WebID protocol for low-level ACL-based protection of database objects (RDF or Relational) and Web Services.	Enables application of sophisticated security and data access policies to Web Services (e.g., SPARQL endpoint) and actual DBMS objects.
Webfinger	Supports using `mailto:` and `acct:` URIs in the context of WebID and other mechanisms, when domain holders have published necessary XRDS resources.	Enables more intuitive identification of people and organizations.
Fingerpoint	Similar to Webfinger but does not require XRDS resources; instea,d it works directly with SPARQL endpoints exposed using auto-discovery patterns in the `<head />` section of HTML documents.	Enables more intuitive identification of people and organizations.

What is the platform?

Tue, 05 Oct 2004 16:31:44 GMT

I came across an interesting piece by Adam Bosworth titled "What is the platform?"

Enterprise Databases get a grip on XML

Tue, 06 Jan 2004 23:17:54 GMT

Databases get a grip on XML
From Inforworld.

The next iteration of the SQL standard was supposed to arrive in 2003. But SQL standardization has always been a glacially slow process, so nobody should be surprised that SQL:2003 ? now known as SQL:200n ? isn?t ready yet. Even so, 2003 was a year in which XML-oriented data management, one of the areas addressed by the forthcoming standard, showed up on more and more developers? radar screens.

Virtuoso Hosting CLR & ASP.NET Demo

Sun, 26 Oct 2003 23:29:19 GMT

I finally have two live servers that demonstrate Virtuoso