[SOLVED] what are my CPUs doing?

[jlittle]

I get these slow downs when my system (Oneiric) really struggles, responding slowly to mouse clicks, maybe 20 or 30 s after the click.

There is an Athlon 64 X2 dual core, 2.2 Ghz, with 1 GiB RAM, usually running firefox with a dozen tabs in Google groups or KFN.

When I've eventually managed to get the K system monitor going, one CPU shows 95 to 100% and the other varying, yet on the process table the top process might be kwin or ksysguard at 4%. The slowdown is like the system is short on memory, yet the process table might show firefox on 300,000 K, plasma desktop on 57,000, Xorg on 44,000, ksysguard and kwin on about 6,000, and 10 more processes with 1 or 2 K. Running top in a konsole shows the same top processes as the system monitor. 4% is not 99%

Following some threads here at KFN, I've disabled Nepomuk and a few others, and this has helped. Until I changed /etc/cron.weekly/apt-xapian-index to use --update the system was effectively useless for a while once a week.

So, what is hogging the cpu and memory?

[whatthefunk]

Maybe its your RAM. Right now, I have nothing open but Opera with 2 tabs (plus I run Cairo dock) and my system is using 1GB RAM. I just opened a dozen tabs with various webpages and now Im at 1.3 GB, which your system would struggle with.

[SecretCode]

If your RAM is used up you could experience swapping. That slows my system down the way you describe.

But so could other types of heavy I/O. If you install and run iotop you can see which processes are performing lots of I/O.

btw, top shows cpu for each process only the "user" time. There's also system time, I/O wait time, and others. Look at the line in top above the processes, something like this:

Cpu(s): 16.8%us, 10.5%sy,  0.0%ni, 72.3%id,  0.3%wa,  0.0%hi,  0.2%si,  0.0%st

If %us or %ni are high, you should see processes with high individual CPU usages. But %sy and %wa (wait) don't necessarily work that way.

[jlittle]

Clearly, my system is swapping because it's short of RAM. But why? What is using the RAM and CPU? Your answer tells me "system time", I'll look more closely at top the next time it happens. Are I/O wait time and the "others" counted as CPU busy by ksysguard? If so, that would explain the CPU. Now I just have to find what's hogging the RAM. Of course, the elephant in the room is firefox, and if it's the culprit my query becomes why does it's memory usage show so low in the process table?

[HalationEffect]

Maybe its your RAM. Right now, I have nothing open but Opera with 2 tabs (plus I run Cairo dock) and my system is using 1GB RAM. I just opened a dozen tabs with various webpages and now Im at 1.3 GB, which your system would struggle with.

Good grief, I wonder what on your system is gobbling up so much RAM?

For comparison right now I have Firefox with 4 tabs, Dolphin (viewing a folder with 'Preview' thumbnails switched on), and VLC playing a video clip, and I'm only using around 900MB of RAM for all that... and roughly 300MB of that 900MB is disk cache...

[whatthefunk]

Good grief, I wonder what on your system is gobbling up so much RAM?

For comparison right now I have Firefox with 4 tabs, Dolphin (viewing a folder with 'Preview' thumbnails switched on), and VLC playing a video clip, and I'm only using around 900MB of RAM for all that... and roughly 300MB of that 900MB is disk cache...

Good question. I killed Cairo-Dock and that bumped off a bit. Right now, I only have Opera running with two tabs open and am using 1 GB. Thats according to the Memory Status desktop widget.

[SecretCode]

@jlittle - yes, the CPU graph in ksysguard includes the system and wait time.

To see definitively what is consuming the most memory, type

ps -e -o pid,etime,comm,%cpu,time,%mem,rss,vsz --sort -rss | head -n 11

At the moment I get the following, showing that firefox is the biggest culprit (which I know), but that Xorg, kwin, plasma-desktop and kded4 (which are all KDE or X system processes) are also using lots of memory.

  PID     ELAPSED COMMAND         %CPU     TIME %MEM   RSS    VSZ
  496  1-00:20:20 firefox          3.5 00:51:24 13.5 549568 1608308
 1338  8-22:27:10 Xorg             6.4 13:54:19 11.9 484464 682156
16566  7-11:42:07 kwin             4.2 07:41:19  4.9 200184 922780
 8998       36:12 java             2.3 00:00:50  4.6 188032 900052
12684  7-22:11:49 plasma-desktop   1.9 03:39:48  4.3 175924 1117280
 1958  8-22:26:20 kded4            0.0 00:04:29  3.2 132096 817816
24973  3-19:51:06 clementine       1.0 00:58:45  2.4 100240 1090760
 2528  8-22:25:48 klipper          0.0 00:00:12  2.3 93876 362404
 8942       37:40 soffice.bin      0.2 00:00:05  1.7 71552 775596
  792    23:57:42 plugin-containe  0.7 00:10:15  1.6 65576 621808

[rms]

I don't know what causes so high cpu & memory usage, but have you tried zram? Some folks reported that it improved system performance even on systems with 1GB RAM.

[SteveRiley]

The philosophy behind zRam/CompCache has always eluded me. If a computer is already memory-constrained, how does taking away some of that memory actually improve anything?

[rms]

The philosophy behind zRam/CompCache has always eluded me. If a computer is already memory-constrained, how does taking away some of that memory actually improve anything?

I'm not sure how it works. It doesn't seem to be using any resources or usage is negligent except that now I have 1GB swap(which is not in existance for Kubuntu).

~$ free -m
             total       used       free     shared    buffers     cached
Mem:          2013       1417        596          0         61        621
-/+ buffers/cache:        734       1279
Swap:         1006          0       1006

734MB used here because Firefox is running for long but when I cold boot it shows its regular ~450MB usage. I'll try to push the system to see how it will do.

[jlittle]

To see definitively what is consuming the most memory, type

ps -e -o pid,etime,comm,%cpu,time,%mem,rss,vsz --sort -rss | head -n 11

Thank you, I've enscripted that and will attempt to run it at my next slowdown.

[SteveRiley]

I'm not sure how it works.

So I decided to investigate this a bit more, and it's smarter than I first thought. The file system it creates in memory is compressed, so the end result appears almost like you added more RAM.

Imagine you have 2 GiB of RAM and that you also have a 2 GiB swap partition. You have a total of 4 GiB program memory -- 2 GiB on fast silicon, and 2 GiB on slow iron. When the silicon fills up but programs still demand more storage, some of what was in silicon migrates to iron. Thus, swap.

Now add zRam to the mix. Create a 512 MiB partition. If we assume that zRam compresses 50%, you've effectively created a 1 GiB swap partition, but it resides in RAM. The performance of the zRam partition will be slightly slower than native memory, becuase of the required compression/decompression. But the end result approximates 2.5 GiB silicon (1.5 GiB remaining plus approx. 1 GiB zRam). And the zRam partition has a higher priority than the disk swap partition, so it's always used first.

Pretty neat. An interesting experiment might be to create your zRam partition to occupy 75% of real RAM, and then set swap priority to 32767 (highest possible). I can imagine how this would provide the experience of getting more memory. It's the compression that matters, and that's what I hadn't thoroughly thought through.

[rms]

Pretty neat. An interesting experiment might be to create your zRam partition to occupy 75% of real RAM, and then set swap priority to 32767 (highest possible). I can imagine how this would provide the experience of getting more memory. It's the compression that matters, and that's what I hadn't thoroughly thought through.

I can add too, from a short testing session, that it seems to be working nicely in Oneiric. It added some extra cpu cycles but overall experience is good for older machines which cannot be extended with more RAM.

[jlittle]

... If we assume that zRam compresses 50%...

Why 50%? I googled and clicked for a while and came up with nearly no idea (one reference to 2x) on this, nor what proportion of one's RAM to devote to it.

I did get the impression that it was a good thing. I'll give it a go when I move to Precise.

[rms]

Why 50%? I googled and clicked for a while and came up with nearly no idea (one reference to 2x) on this, nor what proportion of one's RAM to devote to it.

I did get the impression that it was a good thing. I'll give it a go when I move to Precise.

Yes, it can be more. I have been testing it for two weeks now and never, so far, had any problems whatsoever. Now I can play Amarok and not think about it.