nepomuk db

[geezer]

I have been backing up my home directory and selected other directories to to independent internal hard drives for a while now and usually the back ups take a few seconds, maybe a minute after a long session.

Since upgrading to 10.04, the backups have been taking several minutes. I started watching the file list and the culprit is the nepomuk db. It is 1.2 GB and it seems that even for a sessions of only five minutes requires the whole db to be backed up again.

Is newpomuk essential? I sppent a minute or 2 browsing the web page and it's something to do with meta data that is used for searching.

Is it necessary that the db be 1.2 GB!!!!!!!

What would happen if I un-installed nepomuk?? Would I lose anything essential other than nepomuk?

It seems rather ridiculous to me that it would require a 1.2 GB db - it seems that unless the nepomuk search engine is really, really, really good it would be more efficient for me to just search as I have been for years. I don't do searches for video or images or audio files. I don't do searches for OpenOffice files.

I use the method I have used to decades in searching for relevant files - I use the directory structure and file names.

Would anything drastic happen if just un-installed this beast??

[jglen490]

Re: nepomuk db

Are you running kontact? If so can you live without it, or with some replacement?

It seems that KDE is moving towards akonadi, nepomuk, and related programs as the backend for kontact. So, yes, anytime there is a change to a contact or address book there will be some sort of entry in the database. there may be a way around it, but I just nuked kontact and went with Thunderbird/Lightning for my PIM needs.

There are folks who for one reason or other really need kontact, so I'm not sure how to get through your problem.

[geezer]

Re: nepomuk db

Are you running kontact? If so can you live without it, or with some replacement?

It seems that KDE is moving towards akonadi, nepomuk, and related programs as the backend for kontact. So, yes, anytime there is a change to a contact or address book there will be some sort of entry in the database. there may be a way around it, but I just nuked kontact and went with Thunderbird/Lightning for my PIM needs.

There are folks who for one reason or other really need kontact, so I'm not sure how to get through your problem.

I may have it installed, but I have never knowingly used it. So if it is installed I can get rid of it.

And then get rid of nepomuk also?

That 1.2 GB db is just plain insanity!!!! I'm pretty sure it must be the single biggest file on the system!! And I wasn't even asked if I wanted it!! that is just plain crazy!

[Rog131]

Re: nepomuk db

About Nepomuk (&Strigi)

Few Links:
> aseigo: i don't need no stinking nepomuk .. right?
> What's possible with Nepomuk/Strigi - Few standalone applications


Settings:

System Settings > (Advanced) > Desktop Search - Enable/Disable
System Settings > (Advanced) > Service Manager - Start/Stop Services


Defaults / One Size Fits All ?

a)
With the Kubuntu 10.04 LTS the developers decided that the Strigi indexing is enabled by default. But the Nepomuk & Strigi are not mature, robust applications: Bug #578215: virtuoso-t eats my cpu:

Jonathan Thomas wrote on 2010-06-03:

The solution for now is to disable indexing by default. It has not been enabled in previous releases, and we can address it in kubuntu-default-settings in a way that users who currently have it enabled will not lose their settings.

The underlying issue of virtuoso not having a low enough priority still needs to be addressed, but I'm not as hopeful for an SRU for that bit.
...
Jonathan Thomas wrote on 2010-06-03:

For default settings in maverick, we should try a "wait and see" approach and see if things improve.
I'll attach a patch for kubuntu-default-settings for lucid in a bit here.
...

b)
The user needs to set the limits to the Nepomuk&Strigi (System Settings > (Advanced) > Desktop Search).

At here the Strigi is indexing only limited number of directories, so the disk usage is ok.

[img width=400 height=285]http://img822.imageshack.us/img822/3142/nepomukdiskusage.png[/img]




c)
User should make the decision - do i need the desktop search/idexing - or not.


About disk space

> FAQ: Free Disk Space

Another disk space hog is the thumbnail system. The KDE is not having automatic cleaner.

> Bug 79943 - it should be possible to limit the amount of disk space used by thumbnails (.thumbnails directory)
> Bug 145301 - There is no (GUI?) way to limit thumbnail cache size or age

There is the sweeper:

history and temporary file cleaner for KDE 4

Sweeper can quickly remove temporary information, such as web page cookies,
browser history, or the list of recently-opened documents. It helps provide
additional privacy on a system shared between multiple users.

This package is part of the KDE 4 utilities module.

[img width=400 height=218]http://img31.imageshack.us/img31/3980/sweeper.png[/img]
and/or make an own script to clean the thumbnails.

[jglen490]

Re: nepomuk db

In my opinion, the approach being taken by KDE WRT kontact/akonadi/nepomuk/et al results in far to complex of a solution for most user requirements. I know, based on conversations on the KDE forums, that none of the environment software for kontact is particularly mature and not necessarily stable or at least interoperably stable. They are moving away from kresources to the mix of akonadi/nepomuk/et al. When asked, there was no explanation as to what problem was being solved by the move. The real question as to when all these pieces of the PIM puzzle will be stable and fully interoperable does not seem to be answered.

I know, there are a few people working the KDE DE and apps pretty much on a part time basis. I think they are capable folks, but have bitten off a really big piece of pie. It's going to take a while until the QT4-based software is really up to par and promise.

I like KDE, fundamentally, much more than most other DEs. I just wonder whether there was any user input, or if the KDE 4 requirements were just taken out of the blue in random thoughts. I'm afraid it's more of the latter than former. Too bad.

[geezer]

Re: nepomuk db

Everything I read in what you linked tells me that we have some developers that are running amok. They have decided that they have invented a really, really nice sandbox to play in and in order to justify their sandbox, they have to make everybody use it or at least run it whether they use it or not. Mostly not, since even the developers cannot really define what they are doing or inventing. They sound more like lawyers than s/w developers using very esoteric words which have ambiguous meanings and really do not mean anything but what the speaker wants them to mean. Hence they can use the words to mean whatever they want without ever clearly defining them. Since they cannot even define the words they are using, they cannot define the s/w they are developing. That is amply evident in what they writing to attempt to describe what they are doing.

I left Red Hat because of that reason. I fervently pray that (K)ubuntu doesn't start doing the same thing:

Gee we think this is really nice and its fun to play with, but we don't really know exactly what anybody will use it for, but if we make everybody run it, then hopefully eventually someone will come up with a really nifty app that everybody will want and have to use and that will justify what we have done.

Yeah sure.

When it takes 20 GB memory and 20 TB disks and 50 THZ processor speed to run the silly thing, why should I??

About Nepomuk (&Strigi)

Few Links:
> aseigo: i don't need no stinking nepomuk .. right?
> What's possible with Nepomuk/Strigi - Few standalone applications


Settings:

System Settings > (Advanced) > Desktop Search - Enable/Disable
System Settings > (Advanced) > Service Manager - Start/Stop Services


Defaults / One Size Fits All ?

a)
With the Kubuntu 10.04 LTS the developers decided that the Strigi indexing is enabled by default. But the Nepomuk & Strigi are not mature, robust applications: Bug #578215: virtuoso-t eats my cpu:

Jonathan Thomas wrote on 2010-06-03:

The solution for now is to disable indexing by default. It has not been enabled in previous releases, and we can address it in kubuntu-default-settings in a way that users who currently have it enabled will not lose their settings.

The underlying issue of virtuoso not having a low enough priority still needs to be addressed, but I'm not as hopeful for an SRU for that bit.
...
Jonathan Thomas wrote on 2010-06-03:

For default settings in maverick, we should try a "wait and see" approach and see if things improve.
I'll attach a patch for kubuntu-default-settings for lucid in a bit here.
...

b)
The user needs to set the limits to the Nepomuk&Strigi (System Settings > (Advanced) > Desktop Search).

At here the Strigi is indexing only limited number of directories, so the disk usage is ok.

[img width=400 height=285]http://img822.imageshack.us/img822/3142/nepomukdiskusage.png[/img]




c)
User should make the decision - do i need the desktop search/idexing - or not.


About disk space

> FAQ: Free Disk Space

Another disk space hog is the thumbnail system. The KDE is not having automatic cleaner.

> Bug 79943 - it should be possible to limit the amount of disk space used by thumbnails (.thumbnails directory)
> Bug 145301 - There is no (GUI?) way to limit thumbnail cache size or age

There is the sweeper:

history and temporary file cleaner for KDE 4

Sweeper can quickly remove temporary information, such as web page cookies,
browser history, or the list of recently-opened documents. It helps provide
additional privacy on a system shared between multiple users.

This package is part of the KDE 4 utilities module.

[img width=400 height=218]http://img31.imageshack.us/img31/3980/sweeper.png[/img]
and/or make an own script to clean the thumbnails.

[dibl]

Re: nepomuk db

Heh -- good rant, geezer!

To be brutally honest, you and I have about the same amount of control over the Debian, KDE, and Canonical developers as we do over the Microsoft and Oracle developers. In other words, none. We must sit at our system and pray they send software that is better, in some sense of the word, than what they sent us last year.

[GreyGeek]

Re: nepomuk db

What this world needs is a good 5 cent ciga... opps!, wrong forum!

Actually, the people that influence the developers the most are those who post bug reports, complaints AND wish lists at the bugzillas. For KDE here is the place to post those items. (Be civil! -- most are volunteers. Unless you have the ability to replace them don't drive them off with uninformed rants and personal attacks. You CAN kill the goose that lays the Gold Egg.)

What KDE4 needs is a simple GUI interface to "locate" and "updatedb". "Locate" is good for locating files by name or a piece of the name, but searching inside files it doesn't do. For that, I use Kate, believe it or not. I've opened up over 1,000 text files at once with Kate and selected the Edit-->Find In Files menu option, which opened a dialog into which I put my search pattern. After the search completed (a couple seconds) I had a list of matches showing the line containing the match, and its line number. When I clicked on a line that document opened up in Kate's editor AT the line where the match was found.

[kubicle]

Re: nepomuk db

What KDE4 needs is a simple GUI interface to "locate" and "updatedb". "Locate" is good for locating files by name or a piece of the name, but searching inside files it doesn't do.

Somewhat off-topic, but anyway

kfind can use locate (just check "use file index" in the dialog to use locate database on the search).

Another option for easy locate is the locate kioslave (http://kde-look.org/content/show.php...content=120965), package available from Rog's ppa.

kfind can also search inside files, but for that I usually use grep, example:

grep -rin "search phrase" /mydata/files

will search for "search phrase" (case insensitive) in all files recursively under /mydata/files and output is like:
path/to/file:linenumber:line contents
(You can also use regular expressions with grep searches)

[geezer]

Re: nepomuk db

Yes - grep is my favorite search tool - well actually QTgrep, my own implementation of text and regular expression search that is faster than grep and searches for multiple strings and expressions which is what I usually do when doing such searches. But then 100% of my searches are in text files.

I have never had the necessity to search audio or image files and have absolutely no idea of what "search audio or image files" could possibly mean. Searching tags within the files for dates or size or some other such data could possibly be done, but how would you search images for "certain types of nose"? I imagine some people have thought about that and have ideas for doing it.

I personally just don't have that many image files (family history archive photos that I scanned, but have those organized such that I can find what I want in a reasonable time) and less than 10 audio files.

Do that many people really have that many different files on their computers that they have to frequently search to find the "needle in the haystack"? I can see large companies having to possible do some such things across many employee computers for some reasons (specifically legal reasons ). But individual??I guess I must be missing something because I have really never known individual that needed that capability.

I still think that nepomuk is an unknown solution looking for an uknown problem to solve.

What KDE4 needs is a simple GUI interface to "locate" and "updatedb". "Locate" is good for locating files by name or a piece of the name, but searching inside files it doesn't do.

Somewhat off-topic, but anyway

kfind can use locate (just check "use file index" in the dialog to use locate database on the search).

Another option for easy locate is the locate kioslave (http://kde-look.org/content/show.php...content=120965), package available from Rog's ppa.

kfind can also search inside files, but for that I usually use grep, example:

grep -rin "search phrase" /mydata/files

will search for "search phrase" (case insensitive) in all files recursively under /mydata/files and output is like:
path/to/file:linenumber:line contents
(You can also use regular expressions with grep searches)

[geezer]

Re: nepomuk db

Agreed. But then a good clean rant (I think I kept it clean without getting personal or anything) does wonders for stress level.

I have filed bug notices and comments on bugzilla and specific app web pages and gotten either of 2 responses:

1. no response except maybe the automated response that acknowledges my action and gives me a number to use.

2. any other response has always been of the flavor that my comment is appreciated, but ....... and what follows is a short or long dissertation as to why they would not ever consider incorporating in the app what I have suggested. (Edit: I mis-spoke here - I once commented to the HPLIP developers about keeping the dialog window size and restoring the window size. They kindly replied and said that they would be working to implement the idea and it would appear in a near future release. I'm still waiting, bu at least they were kind enough to like my idea).

After a few such things I just stopped even filing bug notices. When that little window pops up when an app must be terminated because of a bug and would I like to file a report, I have stopped bothering to do so and just close the window and restart the app. Why bother when nothing happens? At least nothing that I have ever been able to discern. I guess that the bugs I encounter are just to weird to happen often and so the are way down on the list to even investigate.

Heh -- good rant, geezer!

To be brutally honest, you and I have about the same amount of control over the Debian, KDE, and Canonical developers as we do over the Microsoft and Oracle developers. In other words, none. We must sit at our system and pray they send software that is better, in some sense of the word, than what they sent us last year.

[dibl]

Re: nepomuk db

Do that many people really have that many different files on their computers that they have to frequently search to find the "needle in the haystack"? I can see large companies having to possible do some such things across many employee computers for some reasons (specifically legal reasons ). But individual??I guess I must be missing something because I have really never known individual that needed that capability.

I still think that nepomuk is an unknown solution looking for an uknown problem to solve.

@geezer, being somewhat of a geezer myself, I long ago learned to organize my data in some kind of hierarchical system that makes sense to me (if no one else), since my data precedes desktop search capabilities by some years, in most cases.

Yes, I have a few files - music, images, and documents -- here's what Dolphin says is in my music backup partition:

[img width=400 height=359]http://img529.imageshack.us/img529/6333/musicbak.png[/img]


But, in fairness, I am aware of at least one Linux developer/supporter who says it is useful for searching in e-mail when looking for a particular tidbit: http://www.sidux.com/index.php?modul...ang=en&sid=585

That's a problem I don't often have, and can't justify launching search services just for the occasional lost item.

I guess it's good that those who collect stuff en masse can have a desktop search capability to find what they need.