Strigi - How to setup & use

[alexandernst]

Hi, I was wondering when will be strigi ready for normal use (without having to install java and make links and so on...). Maybe for 4.4 ?

Thanks

[Rog131]

Re: Strigi

Maybe . Info > Trueg' blog. The last seems to be (about Virtuoso backend):

October 14, 2009

Today nearly all problems are solved...

About Virtuoso:

Virtuoso is a high-performance object-relational SQL database. As a
database, it provides transactions, a smart SQL compiler, powerful
stored-procedure language with optional Java and .Net server-side
hosting, hot backup, SQL-99 support and more. It has all major
data-access interfaces, such as ODBC, JDBC, ADO .Net and OLE/DB.

OpenLink Virtuoso supports SPARQL embedded into SQL for querying RDF
data stored in Virtuoso's database. SPARQL benefits from low-level
support in the engine itself, such as SPARQL-aware type-casting rules
and a dedicated IRI data type.

Trueg:

February 19, 2009

Now back to the real content: A new backend for Soprano which could finally let us drop the sesame2 backend which needs a JVM.

Today I committed the new Virtuoso Soprano backend...

The Soprano version with the standard Karmic is:

:~$ sopranocmd --version
sopranocmd 2.3.1
...
--backend The backend to use when accessing a storage directly and not via the Soprano server.
Possible backends are:
redland, sesame2
...

and that is is too low. Trueg:

Soprano 2.3.63 – that is the magic version number you need to look out for.

Maybe with the Kubuntu 10.04 / KDE 4.4 > Topic: Lucid Lynx 10.04 or with the backports / PPA

[Rog131]

Re: Strigi

... and the Trueg ( > Trueg's Blog) >> Virtuoso – Once More With Feeling:

October 22, 2009

The Virtuoso backend for Soprano and, thus, Nepomuk can be seen as rather stable now. So now the big tests can begin as the goal is to make it the standard in KDE 4.4...

[doctordruidphd]

Re: Strigi

Well, supposedly everything is in place now in lucid:

greenman@Wolfenstein:~$ sopranocmd --version
sopranocmd 2.3.70

greenman@Wolfenstein:~$ locate virtuoso
...
/usr/lib/soprano/libsoprano_virtuosobackend.so
/usr/share/soprano/plugins/virtuosobackend.desktop
...

But no joy:

greenman@Wolfenstein:~$ nepomukserver   
NepomukServer(23548)/nepomuk (server) Nepomuk::Server::enableNepomuk: 
enableNepomuk true                
NepomukServer(23548)/kdecore (KSycoca) KSycocaPrivate::openDatabase: 
Trying to open ksycoca from "/var/tmp/kdecache-greenman/ksycoca4" 
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: "nepomukontologyloader"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: Queueing "nepomukontologyloader" due to dependency "nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: "nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceController::start: 
Starting "nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: "nepomukqueryservice"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: Queueing "nepomukqueryservice" due to dependency "nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: "nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: "nepomukremovablestorageservice"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: Queueing "nepomukremovablestorageservice" due to dependency 
"nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: "nepomukfilewatch"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceManager::Private::
startService: Queueing "nepomukfilewatch" due to dependency "nepomukstorage"
greenman@Wolfenstein:~$ [/usr/bin/nepomukservicestub] 
nepomukstorage(23550)/kdecore (KSycoca) KSycocaPrivate::openDatabase: 
Trying to open ksycoca from "/var/tmp/kdecache-greenman/ksycoca4"
[/usr/bin/nepomukservicestub] virtual bool Soprano::Virtuoso::BackendPlugin::
isAvailable() const could not find Virtuoso ODBC driver
(Soprano::PluginManager) plugin "virtuosobackend" is not available.
[/usr/bin/nepomukservicestub] nepomukstorage(23550)/nepomuk 
(storage service) Nepomuk::Storage::slotNepomukCoreInitialized: 
Failed to initialize nepomuk core
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceController::
slotServiceInitialized: Failed to initialize service "nepomukstorage"
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceController::
stop: Stopping "nepomukstorage"
[/usr/bin/nepomukservicestub] nepomukstorage(23550)/nepomuk 
(storage service) Nepomuk::Core::~Core: Shutting down Nepomuk storage core.
[/usr/bin/nepomukservicestub] nepomukstorage(23550)/nepomuk 
(storage service) Nepomuk::Repository::~Repository: "main"
[/usr/bin/nepomukservicestub] nepomukstorage(23550)/nepomuk 
(storage service) Nepomuk::Repository::close: "main"
Application '/usr/bin/nepomukservicestub' exited normally...
NepomukServer(23548)/nepomuk (server) Nepomuk::ServiceController::
slotProcessFinished: Service "nepomukstorage" went down

I hate these bloody things that run off the edge of the screen...
Note particularly the line:

(Soprano::PluginManager) plugin "virtuosobackend" is not available.

Well, it's there, but I don't know how to get nepomukserver to see it. Anyone got it working?

[Snowhog]

Re: Strigi

Maybe How To Enable Nepomuk/Strigi Desktop Search in Kubuntu 9.10/KDE4.3

It worked for me in Jaunty, which is why I posted it.

[doctordruidphd]

Re: Strigi

Maybe How To Enable Nepomuk/Strigi Desktop Search in Kubuntu 9.10/KDE4.3

It worked for me in Jaunty, which is why I posted it.

Yeah, I saw that, worked great for jaunty & karmic. But Nepomuk in lucid/kde-4.4 now requires the virtuoso backend, won't work with sesame2.

Edit: did a whole bunch of searching on this, and it looks like all the pieces are in place in lucid for it to work. Just won't come together. Hope we can sort this out before kde-4.4 hits karmic.

[Rog131]

Re: Strigi

Hold your horses

The KDE 4.4 is still a BETA 1

The Lucid Lynx is still an ALPHA 1

Earlier > Trueg's Blog >> Virtuoso – Once More With Feeling:

October 22, 2009

The Virtuoso backend for Soprano and, thus, Nepomuk can be seen as rather stable now. So now the big tests can begin as the goal is to make it the standard in KDE 4.4...

I read that as: If everything goes as planned the KDE 4.4 (and Kubuntu 10.04) will have the Soprano backend Virtuoso as default.

If you want to use the Redland or Sesame, you could use them...


About the Nepomuk&Strigi

The > Trueg's Blog has lot of links.

One of them > Akonadi, Nepomuk and Strigi explained.


Where are we now

The Soprano is now capable to use the Virtuoso backend.

Command&output:

locate virtuoso
...
/usr/lib/soprano/libsoprano_virtuosobackend.so
/usr/share/soprano/plugins/virtuosobackend.desktop

will tell that.

The "/usr/lib/soprano/libsoprano_virtuosobackend.so" is from the package:

apt-file search /usr/lib/soprano/libsoprano_virtuosobackend.so
...
soprano-daemon: /usr/lib/soprano/libsoprano_virtuosobackend.so

soprano-daemon is

daemon for the Soprano RDF framework

This package contains the Soprano daemon, D-Bus service, parser
plugins, and a storage plugin for the Redland RDF Application Framework.

> Topic: Lucid Release Schedule, Plans, KDE 4.4
>> Kubuntu Todo tells:

Package virtuoso and get nepomuk working WIP Lex

(WIP = Work In Progress ?)


Maybe the package name will be the soprano-backend-virtuoso (the sesame package is soprano-backend-sesame) or something else. Lucid packages mailing list> Lucid Lynx archive upload notification list >> The Lucid-changes Archives

[GreyGeek]

Re: Strigi

About a month ago there was another thread about strigi consuming 100% of the CPU time. At the time I reported that my 64b KK, fully updated, was having no problems, but I was in error because I "thought" that strigi was running, when it wasn't. I started it and my CPU0 processing shot up to 100%. I let it run for an hour. It spent all that time on a folder with only 272 files, most document files, some very large. But, no files were being accessed so I shut down the processs and uninstalled strigi.

[doctordruidphd]

Re: Strigi

Thanks for the reply.
As I said above, the virtuoso backend is already there.

greenman@Wolfenstein:~$ locate virtuoso
...
/usr/lib/soprano/libsoprano_virtuosobackend.so
/usr/share/soprano/plugins/virtuosobackend.desktop
...

The sesame2 backend does not work in kde-4.4. Saw that in one of Trueg's posts somewhere (did a lot of searching). I can confirm that - when configured exactly as in karmic, it does not produce an index.

The "/usr/lib/soprano/libsoprano_virtuosobackend.so" is from the package

libsoprano4
installed and configured.

soprano-daemon

Well that might be a problem. I don't see that in /usr/bin or /usr/sbin. I'll have to look next time I boot lucid.

But it sounds like the pieces are all there, and this is really a configuration problem.

Edit: sopranod is there, and loads without errors. But still no strigi.

If kde-4.4 is beta, then everything should work, and if it doesn't, the reason why needs to be determined soon.

[Rog131]

Re: Strigi

Once More With Feeling
a.k.a "I want it NOW !!"


What is missing

> Virtuoso – Once More With Feeling

If you stop the nepomuk and start it again:

qdbus org.kde.NepomukServer /nepomukserver org.kde.NepomukServer.quit

nepomukserver 2> /tmp/nepomuk.stderr

The output will tell what is missing:

...
[/usr/bin/nepomukservicestub] virtual bool Soprano::Virtuoso::BackendPlugin::isAvailable() const could not find Virtuoso ODBC driver
...

There is the plugin:

...
/usr/lib/soprano/libsoprano_virtuosobackend.so
/usr/share/soprano/plugins/virtuosobackend.desktop
...

but not the driver - by the Trueg:

Soprano only needs two files: the virtuoso-t binary and the virtodbc_r(.so) ODBC driver

DIY - if you don't want to wait the official Ubuntu/Kubuntu packages


Trueg:

Step 1

Get Virtuoso 5.0.12 from the Sourceforge download page.

I tried with the Karmic (beta ?) to build the Virtuoso - > I have the missing bits.

Copying virtuoso-t to the /usr/bin/
Copying virtodbc_r.so to the /usr/lib/

Stopping the nepomuk and starting it again:

qdbus org.kde.NepomukServer /nepomukserver org.kde.NepomukServer.quit

nepomukserver 2> /tmp/nepomuk.stderr

...
[/usr/bin/nepomukservicestub] Using Virtuoso Version: "5.0.12-rc11.3041-pthreads"
...
" Sat Dec 12 2009
"
[/usr/bin/nepomukservicestub] "21:39:26 OpenLink Virtuoso Universal Server

...

Starting the Nepomuk&Strigi - may take a while.
System Settings > (Advanced) > Desktop Search...




15 minutes later the indexing is done



The commands:

nepomukservicestub nepomukstrigiservice

Service nepomukstrigiservice already running.

qdbus org.kde.NepomukStorage /nepomukstorage org.kde.nepomuk.Storage.usedSopranoBackend

is telling that the virtuosobackend is in use:

virtuosobackend

Some day the official packages will be there...

[doctordruidphd]

Re: Strigi

Success! Well, almost...

Compiled virtuoso, copied over the two files as suggested. Stopped and restarted nepomukserver and strigiservice. That part worked -- started nepomuk without errors, and built an index, where I told it to.

Problem is search doesn't work. Typing in 'nepomuksearchdf' (or anything else) in dolphin simply returns every file in the directory it indexed.

Oh well. Nothing is simple.

[peterbuldge]

Re: Strigi

has anybody figured out a way to get the search working with this stopgap solution yet?

[doctordruidphd]

Re: Strigi

I'm still messing with it, but as of yet, no luck in getting accurate search results.

This is a little disturbing as kde-4.4 is supposedly at beta. One would expect most things to be working at this point, but many things are not. I had to restore my karmic installation back to kde-4.3 because too many things didn't work -- for me, that's a bad sign.

[peterbuldge]

Re: Strigi

after neopmuk/strigi actually started working in 4.3 I sort of fell in love with it... even though it was only using sesame. It's going to be a real drag if it's broken for a while again just because of the move to virtuoso.

[carsy]

Re: Strigi

Once More With Feeling
a.k.a "I want it NOW !!"


What is missing

> Virtuoso – Once More With Feeling

...snip...


Some day the official packages will be there...

Perfect!
Thank you, Rog131

Trying to get Virtuoso to work with Nepomuk, because KMail Contact App requires it (Akonadi). Worked at ODBC stuff...SopranoBackend...several hours of Linux DIY education, yeah! Finally...uh...what about KubuntuForums?

Your instructions worked just right. A few additions for less technicals like me. It took me a while to find the files in question.

After
"Get Virtuoso 5.0.12 from the Sourceforge download page."
and extracting the archive,
virtodbc_r.so is in
.../virtuoso-opensource-5.0.12/binsrc/driver/.libs/

and virtuoso-t (binary) is in
.../virtuoso-opensource-5.0.12/binsrc/virtuoso/

Thanks again Rog131

Virtuoso really speeds things up; still fooling around with search.

Carl