[Bug 1546911] Re: Please recompile sqlite 3.11 with -DSQLITE_ENABLE_FTS3_TOKENIZER

[James Henstridge]

Tyler: no it isn't. The one argument version allows you to query for the
existence of a particular named tokenizer.  The two argument version is
needed to register a new named tokenizer.  When they disabled this they
didn't offer an alternative for fts3/fts4 users, so the documentation
just says to turn the feature back on if you need it, which is a bit
unsatisfying.

It looks like there is a new API to register tokenizers using the new
fts5 API, but that still seems to be under development so the entire
backend is disabled in the current release:

    $ sqlite3 :memory:
    SQLite version 3.11.0 2016-02-15 17:29:24
    Enter ".help" for usage hints.
    sqlite> create virtual table f1 using fts5(a, content='');
    Error: no such module: fts5

It isn't clear this code is at a point where databases would be
compatible release to release, so is probably not appropriate to even
consider yet.

>From a few web searches, I'm guessing this is the reason it was
disabled:

http://chichou.0ginr.com/blog/1336/abuse-sqlite3-ext-to-bypass-php-
security-restrictions

So it is a problem when an application runs untrusted SQL under the
control of the attacker (and in this case, combined with untrusted PHP
code under the control of the attacker).  That seems like a pretty buggy
application to start with.

-- 
You received this bug notification because you are a member of Ubuntu
Foundations Bugs, which is subscribed to sqlite3 in Ubuntu.
https://bugs.launchpad.net/bugs/1546911

Title:
  Please recompile sqlite 3.11 with -DSQLITE_ENABLE_FTS3_TOKENIZER

Status in mediascanner2 package in Ubuntu:
  Confirmed
Status in sqlite3 package in Ubuntu:
  Confirmed

Bug description:
  The recent upload of sqlite 3.11 to xenial-proposed has rendered
  mediascanner2 non-functional.  From the release notes, it seems the
  ability to register new full text search tokenizers has been disabled
  by default:

  http://sqlite.org/releaselog/3_11_0.html

  This means that mediascanner2 fails to open the index.  We can't
  switch to any of the built-in tokenizers because they don't handle CJK
  text, so the only option seems to be to re-enable this functionality
  despite it being a potential security vulnerability for apps that let
  untrusted code run arbitrary SQL.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/mediascanner2/+bug/1546911/+subscriptions