High CPU load during startup and sometimes after

[PraysToPan]

You know, there's no doubt that wasting a few cpu cycles on logic to schedule IO efficiently, is worth it. I think it's getting to the point that a little bit of automation needs to be applied to tuning memory allocation, too. If not, problems like these are never going to end. Distributions will have no choice but to 'choose a side' for configurations. Inflexible kernels are the norm. That's always going to leave a subset of users with a bad taste.

[SteveRiley]

It does seem like the decision to move to deadline for desktops/laptops is a little weird, especially for those with spinning disks. I wonder how many other distros have done this. All this effort to "streamline" configs is looking more and more like laziness than anything else.

[PraysToPan]

What processor do you have in your laptop? On my D420 with the Core 2 Duo, there is only 2MB of cache. It's quite easy to have it constantly invalidated.

These sorts of problems are only going to get worse, IMO. Just consider where hardware has been heading: tablets. It's getting to the point that most everyone expects the same functionality with them as they do on PCs, yet they are more lightly configured than the old notebook I had the trouble with. Add that to the fact that wasted CPU ruins battery life. If Ubuntu and other distributions want to see people choosing to install on these devices, the kernel will have to become a little smarter, methinks.

[Snowhog]

A bit of 'history': Deadline as default scheduler?

On my newer HP laptop, running 12.04 (3.2.0-39-generic 64-bit, i3 processor) w/4GB RAM I have: CONFIG_DEFAULT_IOSCHED="cfq"
On my older Toshiba, running 12.10 (3.5.0-26-generic 32-bit, dual-core T2060) w/2GB RAM I have: CONFIG_DEFAULT_IOSCHED="deadline"

[PraysToPan]

Oh, I know. That's one of the two items in the title of my bug report. https://bugs.launchpad.net/ubuntu/+s...x/+bug/1159258 Switching my notebook back to CFQ was my first approach. It helped a somewhat; maybe 10 to 20 %. It was transparent hugepages that really made the difference, though.

So, I'm curious, did you find much of a performance drop on your Toshiba switching from 12.04 to 12.10, assuming you ran both on them on it?

[Snowhog]

I don't have any performance problems on either laptop; never have. The Toshiba boots quicker than the HP; it presents the Grub menu faster.

[SteveRiley]

What processor do you have in your laptop? On my D420 with the Core 2 Duo, there is only 2MB of cache. It's quite easy to have it constantly invalidated.

HP Mini 2140:

root@mini2140:~# [B]cat /proc/cpuinfo[/B]
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 28
model name      : Intel(R) Atom(TM) CPU N270   @ 1.60GHz
stepping        : 2
microcode       : 0x208
cpu MHz         : 800.000
cache size      : 512 KB

ThinkPad T520:

steve@t520:~$ [B]cat /proc/cpuinfo[/B]
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model  : 42
model name : Intel(R) Core(TM) i7-2620M CPU @ 2.70GHz
stepping : 7
microcode : 0x18
cpu MHz  : 800.000
cache size : 4096 KB

These sorts of problems are only going to get worse, IMO. Just consider where hardware has been heading: tablets. It's getting to the point that most everyone expects the same functionality with them as they do on PCs, yet they are more lightly configured than the old notebook I had the trouble with. Add that to the fact that wasted CPU ruins battery life. If Ubuntu and other distributions want to see people choosing to install on these devices, the kernel will have to become a little smarter, methinks.

How much automatic platform detection and behavior adjustment do we want in the kernel itself? Is that really the kernel's job? Maybe a smarter udev might work. Interesting to ponder, regardless.

[PraysToPan]

You just mentioned the key problem: streamlining. I'm sure deadline works fine for a great number of users with the right hardware. A beefy multicore CPU with large cache would most likely get the best latency out of it. A loss of a bit of raw processing efficiency wouldn't be worth worrying about as long as your latest 3D shooter got a high frame rate.

I don't know all that much about the scheduler algorithms. It did occur to me that if deadline might be so rigid, perhaps it's faulting a process part way through a block. If this is the case, that's just more pressure on the CPU cache because it would be cleared out as the next process is running. A more forgiving scheduler could be smart enough to allow the block to be handled within a single context.

[PraysToPan]

Not a bad idea. I did just that, using noop, for my horrible SD card slot.

[PraysToPan]

I'm just saying that it's needed for broader markets. I'm fine enjoying myself with some performance tuning, but most people aren't.

Can you imagine the average Windows user trying it?

[SteveRiley]

Can you imagine the average Windows user trying it?

Not necessary. Windows uses a completely different mechanism for I/O scheduling, based on assigning I/O priorities to individual processes. This concept was introduced in Vista and Server 2008, and continues in 7, Server 2008 R2, 8, and Server 2013.

http://en.wikipedia.org/wiki/Windows...O_technologies
http://technet.microsoft.com/en-us/m...stakernel.aspx
http://msdn.microsoft.com/en-us/wind.../gg454515.aspx

[PraysToPan]

You're quite right about managing a paging strategy i in userspace though. It doesn't seem like a hard task at all. Keep an eye on CPU cache and translation lookahead buffer misses and turn it on or off as needed. Perhaps cgroups could be used to enforce the proper strategy on heterogeneous work loads on the same system.

[SteveRiley]

Perhaps cgroups could be used to enforce the proper strategy on heterogeneous work loads on the same system.

I would suggest that you join LKML and ask if anyone's thought of this. Seems like a neat idea, but I don't have my head sufficiently wrapped around cgroups just yet to ponder possible implementation details.

[PraysToPan]

Transparent hugepages isn't quite there yet, though. It provides khugepaged to merge 4k pages into hugepages on the fly at times like context switches and page faults. It also turns them back. into 4k pages before they are swapped since swap is 4k only. Unfortunately, from what I read, it only optimizes tasks that start after it does. Turning it off means you might not get the benefit back again when you need it.

Anyhow, thanks for the great chat.

Time to get back to the work I'm actually paid for!

Cheers,
Jon

[SteveRiley]

LOL, I just had to explain all this stuff to my wife -- sometimes she asks what I'm doing on here, and I'll tell her! We decided that men and women implement the following queueing algorithms:

* Men speaking to men: noop, because men don't need anything more than a simple FIFO buffer
* Women speaking to women: CFQ, of course, because women are always completely fair with each other
* Women speaking to men: deadline, because women will wait only a short finite time before forcibly changing the priority of men

Bwhahahaha.