[KDE] Performance Issues?

[PhysicistSarah]

I'm having some performance issues that I'm not sure what the cause is.

Even when the system is ideling, the CPU is usage is high. can't find the origin of the issue though from system monitor. When I go t the processes tab -> all processes -> sort by CPU usage, it shows some process using soething like 3% and nothing else. The rest of the system experiences freezes all the time, in which I can't do anything and the sccreen doesn't update, then it returns to normal before freezing few seconds later.

I'm not sure of the cause of this behaviour or how to diagnose a particular cause. I supect a kernal update (I did go though a few kernal updates), but I'm not sure. Also, when the system is lagging like this, the keyboard mistypes chiaracters occasionally It might just throw random character that I didn't type into the sentance for some reason.

System monitor during idle:

Here is a hardinfo report:
https://www.dropbox.com/s/ptqsq77psf...port.html?dl=0

[chimak111]

You could install inxi and post the output of inxi -Fxxxz here.

[GreyGeek]

My first thought when I see something like this is the baloo indexing gone wild. Baloo indexing can be disabled by
balooctl disable
followed by a reboot.
More info here, which included how to set ballorc to shut off indexing

The fact that the processes tab shows nothing going over 3% is odd. It suggests that something is running but concealing its presence from the process id list. I noticed in your list of directories that you have one labled ELYSIUM. Is that related to a Steam game or to bitcoins? Do you have a bitcoin miner running in the background?

[PhysicistSarah]

@GreyGeek After disabling balooctl, the issue stopped and CPU usage has returned to normal.

Does this mean that I'll lose indexing for search though? Is there any way to resolve this without losing indexing? (Like resetting a cache or something to stop it from freaking out?)

ELYSIUM is the name for a hard drive I recovered from an older computer that was salvaged for parts.

I do not have a bitcoin miner because I don't currently own a server farm or power plant

EDIT: It looks like the issue still persists actually. CPU usage just jumped while nothing is happening. I checked baloo status, it's offline after the computer rebooted.

[oshunluvr]

Usually once baloo indexes everything it will calm down. You might enable it when you're not using the computer and let it run.

Also, come process may be "bouncing" usage and not showing up in the GUI due to response times of the graphs or maybe it's outside the parameters for the GUI tool. Try running "top" in a terminal and watching during the high-CPU periods. It could easily be plasma or some drive format problem.

[PhysicistSarah]

sarah@ConvergentRefuge:~$ inxi -Fxxxz
System: Host: ConvergentRefuge Kernel: 4.15.0-33-generic x86_64 bits: 64 compiler: gcc v: 7.3.0
Desktop: KDE Plasma 5.12.6 tk: Qt 5.9.5 wm: kwin_x11 dm: sddm
Distro: Ubuntu 18.04.1 LTS (Bionic Beaver)
Machine: Type: Laptop System: Hewlett-Packard product: HP ENVY 17 Notebook PC v: 097E110000405E00000620100
serial: <filter> Chassis: type: 10 serial: <filter>
Mobo: Hewlett-Packard model: 1968 v: KBC Version 93.52 serial: <filter> UEFI: Insyde v: F.68
date: 07/22/2016
Battery: ID-1: BAT0 charge: 48.6 Wh condition: 48.6/48.6 Wh (100%) volts: 12.3/10.8
model: Hewlett-Packard Primary type: Li-ion serial: <filter> status: Full
CPU: Topology: Quad Core model: Intel Core i7-4702MQ bits: 64 type: MT MCP arch: Haswell rev: 3
L2 cache: 6144 KiB
flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx bogomips: 35117
Speed: 1913 MHz min/max: 800/3200 MHz Core speeds (MHz): 1: 1913 2: 1505 3: 2070 4: 1950 5: 1960
6: 1895 7: 1864 8: 1880
Graphics: Card-1: Intel 4th Gen Core Processor Integrated Graphics vendor: Hewlett-Packard driver: i915
v: kernel bus ID: 00:02.0 chip ID: 8086:0416
Card-2: NVIDIA GK107M [GeForce GT 750M] vendor: Hewlett-Packard driver: nvidia v: 396.54
bus ID: 01:00.0 chip ID: 10de:0fe4
Display: x11 server: X.Org 1.19.6 driver: modesetting,nvidia unloaded: fbdev,nouveau,vesa
compositor: kwin x11 tty: N/A
OpenGL: renderer: GeForce GT 750M/PCIe/SSE2 v: 4.6.0 NVIDIA 396.54 direct render: Yes
Audio: Card-1: Intel Xeon E3-1200 v3/4th Gen Core Processor HD Audio vendor: Hewlett-Packard
driver: snd_hda_intel v: kernel bus ID: 00:03.0 chip ID: 8086:0c0c
Card-2: Intel 8 Series/C220 Series High Definition Audio vendor: Hewlett-Packard
driver: snd_hda_intel v: kernel bus ID: 00:1b.0 chip ID: 8086:8c20
Sound Server: ALSA v: k4.15.0-33-generic
Network: Card-1: Intel Wireless 7260 driver: iwlwifi v: kernel bus ID: 07:00 chip ID: 8086:08b1
IF: wlp7s0 state: up mac: <filter>
Card-2: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet driver: r8169 v: 2.3LK-NAPI
port: 3000 bus ID: 0f:00 chip ID: 10ec:8168
IF: eno1 state: down mac: <filter>
IF-ID-1: tun0 state: unknown speed: 10 Mbps duplex: full mac: N/A
Drives: Local Storage: total: 1.06 TiB used: 805.72 GiB (74.1%)
ID-1: /dev/sda vendor: Toshiba model: MQ01ABD100 size: 931.51 GiB speed: 3.0 Gb/s rotation: 5400 rpm
serial: <filter> rev: 1C temp: 44 C scheme: GPT
ID-2: /dev/sdb vendor: LITE-ON IT model: LMS-24L6M-HP size: 22.37 GiB speed: 6.0 Gb/s
serial: <filter> rev: 704 scheme: GPT
ID-3: /dev/sdc type: USB model: SABRENT SABRENT size: 111.79 GiB serial: <filter> rev: 0204
scheme: GPT
ID-4: /dev/sde type: USB vendor: SanDisk model: Gaming Xbox 360 size: 7.48 GiB serial: <filter>
rev: 8.02 scheme: MBR
ID-5: /dev/sdf type: USB vendor: Toshiba model: TransMemory size: 14.92 GiB serial: <filter>
rev: 1.00 scheme: MBR
RAID: Hardware-1: Intel 82801 Mobile SATA Controller [RAID mode] vendor: Hewlett-Packard driver: ahci
v: 3.0 port: 6060 bus ID: 00:1f.2 chip ID: 8086.282a rev: 05
Partition: ID-1: / size: 775.55 GiB used: 726.95 GiB (93.7%) fs: btrfs dev: /dev/sda2
ID-2: /home size: 775.55 GiB used: 726.95 GiB (93.7%) fs: btrfs dev: /dev/sda2
Sensors: System Temperatures: cpu: 60.0 C mobo: 60.0 C gpu: nvidia temp: 56 C
Fan Speeds (RPM): N/A
Info: Processes: 421 Uptime: 7h 25m Memory: 15.60 GiB used: 12.79 GiB (82.0%) Init: systemd v: 237
runlevel: 5 Compilers: gcc: 7.3.0 alt: 7 Shell: bash v: 4.4.19 running in: konsole inxi: 3.0.20

[PhysicistSarah]

I found out about "iotop" and was able to determine the cause of the disturbance. It turns out that it was a bug in timeshift doing snapshots when I didn't want it to and bringing the system to a halt.

[GreyGeek]

I found out about "iotop" and was able to determine the cause of the disturbance. It turns out that it was a bug in timeshift doing snapshots when I didn't want it to and bringing the system to a halt.

Interesting!
Many people believe that one can take hundreds of Btrfs snapshots without them affecting performance. The Btrfs devs, on the other hand, recommend making no more than a dozen or so snapshots per subvolume. How many shapshots had TimeShift made during its mad rampage?

It relates to the fact that while a new snapshot is essentially empty, the longer it exists it continues to accumulate changes made to the @ or @home and eventually fills up and in size becomes equal to its parent. If you have a 1TB HD and your @ + @home = 100GB then you can make 8 snapshot pairs (@yyyymmdd + @homeyyyymmdd) before your total usage climbs to about 90% or more. When that happens Btrfs does LOTS of tree walks looking for free chunks.

The problem is explained here:

Having many subvolumes can be very slow

The cost of several operations, including currently balance, device delete and fs resize (shrinking), is proportional to the number of subvolumes, including snapshots, and (slightly super-linearly) the number of extents in the subvolumes.


This is "obvious" for "pure" subvolumes, as each is an independent file tree and has independent extents anyhow (except for ref-linked ones). But in the case of snapshots metadata and extents are (usually) largely ref-linked with the ancestor subvolume, so the full scan of the snapshot need not happen, but currently this happens.


This means that subvolumes with more than a dozen snapshots can greatly slow down balance and device delete. The multiple tree walks involve both high CPU and IOPS usage. This means that schemes that snapshot a volume periodically should set a low upper limit on the number of those snapshots that are retained.

TimeShift has another problem I pointed out in my Beginners Btrfs Tutorial Guide: It combines the <ROOT_FS>with / and /home in such a way that if you remove TimeShift BEFORE you use it to delete ALL your existing snapshots made using it, the results will be the corruption or your <ROOT_FS> and a failure to boot properly at the next power cycle.

What to do?
1. Use TimeShift to delete ALL the snapshots it made.
2. sudo apt purge timeshift
3. Remove ALL snapshots you made manually (You have sent a snapshot of @ and @home to an external storage device? If not do so before continuing)
4. Run the balance operation at least three times in increasing depth:
a. sudo -i (in a Konsole)
b. mount /dev/disk/by-uuid/uuidofdevice /mnt (get the uuid of your btrfs HD by using blkid. THe first UUID is the correct one.)
c. btrfs balance start -dusage=0 /mnt (-d is data, -m is metadata, -s is system)
d. Repeat step c, increasing -dusage by 5 each time until you get to -dusage=50 (It will go quicker than you think!)
e. After getting cleaned up do the following balance command once a week, if not once a day:
btrfs balance start -dusage=25 -dlimit=10 -musage=25 -mlimit=10 /mnt
That balance command can be run daily as a cron job in the background and doesn't hurt performance while you are using your system. I also have a cron job for running scrub once a week.

In my tutorial I also discuss Snapper and its problems. I don't recommend Snapper for other reasons. What do I use to make snapshots? The Konsole and keyboard. I've been threatening to write a bash script to do it but the process is so easy that just do it manually and move on.

IF I were still programming I'd take a set of snapshots: one each noon and evening, deleting the oldest two of the set while keeping the four most resent pair. For a work week that would amount to 20 snapshots, i.e., 10 pairs of @ and @home snapshots. And, I can always take an emergency snapshot pair before committing a large update coming down the pipe. If things run well I can delete the oldest pair and keep the newest pair, otherwise just roll back to the most recent pair.

[oshunluvr]

If you want to "timeshift" it's much safer to do it yourself with a script and anacron. That way you're in control.

[PhysicistSarah]

Thankfully, I took your advice and decided to not use timeshift with BTRFS snapshots. I would never use timeshift in btrfs mode for a lot of reasons. The most notable reason being that it is on the same disk and not an external one. Disk failures are rare, but I'm not about to lose everything if one occurs. I was using timeshift in rsync mode.

I still have to learn how to properly use btrfs though. I kind of reinstalled my system a while back with btrfs and forgot about it. This kind of screwed me, because Dropbox (Which I use Heavily) just announced that they'll only be supporting ext4 from now on. I'll have to check out your tutorial on it @Grey

[GreyGeek]

....
I still have to learn how to properly use btrfs though. I kind of reinstalled my system a while back with btrfs and forgot about it. This kind of screwed me, because Dropbox (Which I use Heavily) just announced that they'll only be supporting ext4 from now on. I'll have to check out your tutorial on it @Grey

Like I used to tell my physics students: "Everything is easy once you know how!" Learning to maintain a Btrfs system is a pretty easy thing to do. At 77 my memory is just a memory, and I keep a printout of the steps I use to create, delete and send snapshots on the wall next to my computer as a handy reference.

I read about what DropBox did.
https://www.dropboxforum.com/t5/Sync...hy/td-p/290058

The supported file systems are NTFS for Windows, HFS+ or APFS for Mac, and Ext4 for Linux.

Creating an EXT4 partition and then mounting it to a directory, say, /ext4, doesn't pass muster with DropBox because, apparently, it still sees Btrfs between it and the EXT4 drive. DropBox claims it " relies on extended attributes (X-attrs) to identify files in the Dropbox folder and keep them in sync" BUT, BTRFS is the standard default file system for ubuntu system partitions and supports the mentioned X-attrs attributes. In fact, the xattr man page states:

In the Btrfs, XFS, and Reiserfs filesystem implementations, there is no practical limit on the number of extended attributes associated with a file, and the algorithms used to store extended attribute information on disk are scalable.

But, in fact, many who are running EXT4 systems, with and without encryption, are getting the notice about service stopping in November.

So, lack of "xattr support", or the presence of encryption, is not the reason. What is? Many are suggesting that DropBox wants to monetize your data.

There are alternatives to DropBox that are less intrusive or restrictive:
https://www.cloudwards.net/top-10-se...-alternatives/
Combining sync.com or pCloud with syncthing as an alternative.

[blackjack]

TimeShift has another problem I pointed out in my Beginners Btrfs Tutorial Guide: It combines the <ROOT_FS>with / and /home in such a way that if you remove TimeShift BEFORE you use it to delete ALL your existing snapshots made using it, the results will be the corruption or your <ROOT_FS> and a failure to boot properly at the next power cycle.
.
.
.
In my tutorial I also discuss Snapper and its problems. I don't recommend Snapper for other reasons. What do I use to make snapshots? The Konsole and keyboard. I've been threatening to write a bash script to do it but the process is so easy that just do it manually and move on.

Any advice for a gui app (for btrfs)for snapshots for us noobs. ? also ive noticed my laptop battery life is worse under btrfs than ext4. this after a fresh install and no timeshift etc installed? What might be the reason?

[jlittle]

Any advice for a gui app (for btrfs)for snapshots for us noobs. ?

I like snapper and snapper-gui, but the btrfs experts here don't much. The default snapper configs make too many snapshots, but after they're turned down a bit snapper just does its thing without trouble, and automatically does regular snapshots; you don't have to remember to do it. I just use the gui mostly to see what's there and to delete snapshots after a backup to external drives, I haven't explored the functionality much.

Regards, John Little

[oshunluvr]

Any advice for a gui app (for btrfs)for snapshots for us noobs. ? also ive noticed my laptop battery life is worse under btrfs than ext4. this after a fresh install and no timeshift etc installed? What might be the reason?

I'm probably the most active BTRFS user here but I don't use the snapshot GUI's. Time past they were a nuisance so most of us left them behind. Honesty I haven't revisited any of them so give it a go and let us know what you think. If you're just looking to make it easier, I wrote a set of Dolphin servicemenus that make manually doing it easier by using Dolphin. I have a newer version than the one in the KDEstore so if you are interested, let me know and I'll post the latest version.

Personally, I am currently using a cron job to take an automatic daily snapshots. I'm working on an automatic backup script as well.

[oshunluvr]

I went ahead and updated the Service menu https://www.opendesktop.org/p/1214134/