The numbering scheme at the core of eXist 1.0 did limit the maximum size of a document to be stored within the database. The limit was not a question of the document size alone, but also depended on the overall number of nodes and how deep elements were nested within the document. The actual limit thus differed from document to document and was difficult to compute in advance. eXist 1.1 fixes this.

Concerning collection management, there are two known problems:

1. if a huge number of documents (> 100,000) is stored in a collection, queries across the whole collection will spend a major part of the query time just for loading the document descriptors
2. the current collection cache has a too small default setting (128) and is currently static. Using more than a few 100 collections will result in a lot of disk I/O and makes the db slow

The first problem could be solved by using placeholder (or proxy) objects for the real document during query processing. Most of the administrative metadata (creation-time etc.) is not required. As of May 2006, the storage location of the document metadata has already been separated from the collection store. However, eXist will still load all the document descriptors when the collection is accessed the first time.

The switch to the DLN numbering scheme also results in smaller document objects. TODO: re-check memory consumption with DLN scheme.

For the second issue, a better, more dynamic caching mechanism for collections needs to be implemented.

Metadata could include system properties like last-modification date or user-defined metadata. Preferably, metadata records should be ordinary XML documents. The format should not be restricted.

Currently, eXist supports these index types:

• structural index
• full text index
• range index
• combined range and qname index
• index on xml:id's

However, there will be other index types added in the future, for example:

• an n-gram based index to better support languages for which word based tokenization is not applicable
• combined path indexes to speed up frequently-used XPath expressions
• indexes to support operations on atomic-values, e.g. in order-by expressions or the distinct-values function
• integration of specialized, external index types (e.g. spatial indexes)

In the current architecture, integration of new index types faces two major problems:

1. there's no general interface to plug in new indexes
2. index selection is hard-coded into the query engine

Preferably, it should be possible for the query engine to determine if an index can be used or not at compile time, i.e. during query analysis phase. To achieve this, index configuration probably needs to be simplified, so the query engine can make a decision on index usage before the query is actually evaluated. It might not be possible to keep the current, fine-grained configuration scheme, which allows indexes to depend on the presence of ancestor nodes. Some commercial (and highly efficient) xml DBMS allow users to define indexes only on a given QName, not a path expression. This topic needs further discussion in the community.

The interfaces to the indexing system need to be redesigned to support the query engine in index selection. This includes, for example, statistical information about the frequency of index items. The redesign of the indexing system thus presents a necessary foundation for the query optimizer.

(Align with the XQuery Full-text specification)

The interfaces to the indexing system need to be redesigned to support the query engine in index selection. This includes, for example, statistical information about the frequency of index items. The redesign of the indexing system thus presents a necessary foundation for the query optimizer.

The current architecture is also too limited with respect to text analysis. The general-purpose tokenizer is not suitable for language-dependent analysis. Plans are to replace these classes by Lucene's analyzer. Lucene offers a pluggable architecture in which multiple analyzers can be combined.

No remarks available.

Works, but special functions need to be used.

Currently stored in the structural index. Should be moved to the range index.

When dealing with texts in many non-European languages, the token-based full-text index produces insufficient results. Tokenization is currently based on Unicode code points. Most chinese characters, for example, are thus stored as single tokens. Users have to abuse the near() or phrase() function to search for character sequences consisting of more than one character, which is quite slow. It also means that real proximity searches are not available.

An N-gram based index would be much more suitable for these languages. It would also allow additional functionality to be implemented, e.g. to deal with varying spellings. The main question is how the N-gram index would integrate conceptually with the existing full-text functions.

For some types of data, e.g.. spatial information, specialized indexes might be provided by other systems.

Order-by expressions and other functions that need to access atomized nodes are not supported by indexes.

The journal log and the recovery manager should be stable and are covered by extensive tests. However, recovery failures can not be excluded entirely. The tests can't reproduce every possible real-world scenario.

However, some steps remain for eXist to become a fully transactional database system. Transaction support is currently limited to the functionality needed for crash recovery. Though we maintain transactions internally, they are currently not exposed to applications. Also, read operations are not transactional right now.

In order to allow user-defined ACID transactions with support for rollback, all index files would need to be protected by the journaling log. The required functionality is basically available, but the feature is currently not regarded as high-priority.

No remarks available.

No remarks available.

Transactions are maintained internally, but they are not exposed to applications. eXist does not yet support full ACID transactions. Read-only operations bypass the transaction system.

Journal logs are limited to critical data required for recovery. No transaction rollbacks.

No remarks available.

No remarks available.

W3C is working on an update extension, but no draft released so far.

The currently implemented Unix-like access control scheme is sufficient to protect resources and collections in a multi-user environment. However, it might be too coarse-grained for some types of applications. A more dynamic ACL implementation could help here. Right now, security management forms part of the database core. This is unnecessary. A more modular architecture would allow different security managers to be plugged in. It would be the responsibility of the security manager implementation to handle ACL lists.

More important, some critical areas are currently not protected: This includes access to stored XQueries, system-critical XQuery functions and the Java binding. Basically all Java classes can be used from within an XQuery and this leaves doors wide open on all systems that allow users to execute XQuery on the server.

Required steps:

• restrict the right to execute stored queries to selected users
• limit access to defined XQuery functions and modules
• fine-grained access control for the Java binding. Usage of Java classes should be disallowed by default. Access to specific classes needs to be allowed by the dba admin

No remarks available.

Need more dynamic ACL structures that can adapt to varying requirements.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

No remarks available.

The XQuery engine as well as the standard function libraries should be updated to align with the latest candidate recommendation. Basically, almost all core language features are implemented, excluding schema related features, which are currently beyond eXist's scope.

XQuery support in eXist is not sufficiently covered by the test suite. In particular, we lack tests for the function library. Implementing the official XQTS XQuery test suite should thus be a top priority in order to guarantee standard conformance and avoid future regressions.

Updated to latest Candidate Recommendation. Stable, excluding schema-related features

Add gregorian dates and NOTATION

Updated to latest Candidate Recommendation. Stable, excluding schema-related features.

Updated to latest Candidate Recommendation

Though we implement most of the serialization options specified in the XQuery and XSLT serialization spec, some options need to be reworked and should be covered by tests.

Should be implemented to avoid future regressions and ensure conformance

Most query optimizations are currently hard-coded into the query engine and are applied at execution time, not at compile time. This puts a limit on extensibility and possible optimizations. As described above, adding new index types is difficult. For example, the operator implementation for “=” and “eq” does currently include about 4 or 5 different optimization paths. Location steps like child::foo can choose among 3 execution alternatives. This makes it hard to see if the correct optimization is applied or not. Adding further index types will make the code unmanageable.

As already explained above, the decision which index to use should thus be moved into the query analysis phase. The index implementation thus has to provide sufficient information to make the decision.

A good part of the performance problems eXist currently has could be solved by a post-compilation/pre-evaluation optimizer and intelligent query rewriting. In particular, query rewriting could be used to reduce the general size of the node sets that need to be processed by an XPath expression, which can result in a tremendous performance boost for queries on huge document sets, if those queries involve a predicate expression which limits the number of potential matches.

Error reports by the XQuery parser and compiler need to be improved.

Recursive functions may trigger a StackOverflowException. We need to handle tail-recursion.

XInclude expansion happens at serialization time. Queries across the included document fragments are not possible. Stable, but limited

No remarks available.

New implementation should be based on REST to avoid current problems (e.g.. With character encodings)

Exposes the entire database functionality.

Does not cover user-management and permissions as well as other administrative functions. Stable, but further functionality could be exposed

General functionality tests required

Could be a simpler alternative to the now somewhat bloated XML:DB API

Provide translations for error messages, console outputs etc. At least, resource bundles should be used, so others can translate them if they want.

All libraries included with eXist need to be checked.

Should explain in more depth how one can write webapps in XQuery, using the XQueryGenerator with Cocoon or stored XQueries.

calling XQuery scripts stored in the DB; import stored modules into a query passed to the DB

No remarks available.

Integration with a servlet engine, Cocoon, stand-alone server, embedded use.

No remarks available.

No remarks available.

The documentation is currently not searchable, though most of it is provided in XML.

Migrate the function documentation to XQDoc. Use XQDoc to better document all XQuery examples.