[SOLVED] Hard drive spindown: root drive wakes frequently

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

I've let it run overnight. I checked the output of btrace. It's weird... exactly every hour, syslogd wakes the drive. It's weird because
1) it can't be the cache flushing, because I set that to 5 hours:

cristian@server:~$ cat /proc/sys/vm/dirty_expire_centisecs
1800112
cristian@server:~$

2) syslogd should write to /var/log, right? And now /var/log is in RAM. It would be nice to know which files are being written, but btrace doesn't say.
Here's the output of one of those "hourly sessions":

 8,0  0  4091 3915.026812622 4732 A W 1508511 + 8 <- (8,1) 1508448
 8,0  0  4092 3915.026816813 4732 Q W 1508511 + 8 [syslogd]
 8,0  0  4093 3915.026835531 4732 G W 1508511 + 8 [syslogd]
 8,0  0  4094 3915.026849221 4732 P N [syslogd]
 8,0  0  4095 3915.026855647 4732 I W 1508511 + 8 [syslogd]
 8,0  0  4096 3915.026903980 4732 A W 524927 + 8 <- (8,1) 524864
 8,0  0  4097 3915.026907612 4732 Q W 524927 + 8 [syslogd]
 8,0  0  4098 3915.026921022 4732 G W 524927 + 8 [syslogd]
 8,0  0  4099 3915.026927727 4732 I W 524927 + 8 [syslogd]
 8,0  0  4100 3915.026940858 4732 U N [syslogd] 2
 8,0  0  4101 3915.026962091 4732 D W 1508511 + 8 [syslogd]
 8,0  0  4102 3915.027069094 4732 D W 524927 + 8 [syslogd]
 8,0  0  4103 3920.267825750  0 C W 1508511 + 8 [0]
 8,0  0  4104 3920.268825934  0 C W 524927 + 8 [0]
 
[...] (the block above repeats for 5 times, always in the 3920 second)

 8,0  0  4161 3925.272156977 136 A W 63 + 8 <- (8,1) 0
 8,0  0  4162 3925.272160888 136 Q W 63 + 8 [pdflush]
 8,0  0  4163 3925.272174857 136 G W 63 + 8 [pdflush]
 8,0  0  4164 3925.272185474 136 P N [pdflush]
 8,0  0  4165 3925.272190502 136 I W 63 + 8 [pdflush]
 8,0  0  4166 3925.272212574 136 A W 71 + 8 <- (8,1) 8
 8,0  0  4167 3925.272216206 136 Q W 71 + 8 [pdflush]
 8,0  0  4168 3925.272225425 136 M W 71 + 8 [pdflush]
 8,0  0  4169 3925.272240232 136 A W 524351 + 8 <- (8,1) 524288
 8,0  0  4170 3925.272243585 136 Q W 524351 + 8 [pdflush]
 8,0  0  4171 3925.272252246 136 G W 524351 + 8 [pdflush]
 8,0  0  4172 3925.272258671 136 I W 524351 + 8 [pdflush]
 8,0  0  4173 3925.272275434 136 A W 524359 + 8 <- (8,1) 524296
 8,0  0  4174 3925.272279066 136 Q W 524359 + 8 [pdflush]
 8,0  0  4175 3925.272286051 136 M W 524359 + 8 [pdflush]
 8,0  0  4176 3925.272299740 136 A W 524527 + 8 <- (8,1) 524464
 8,0  0  4177 3925.272303931 136 Q W 524527 + 8 [pdflush]
 8,0  0  4178 3925.272312313 136 G W 524527 + 8 [pdflush]
 8,0  0  4179 3925.272319018 136 I W 524527 + 8 [pdflush]
 8,0  0  4180 3925.272336898 136 A W 524687 + 8 <- (8,1) 524624
 8,0  0  4181 3925.272340530 136 Q W 524687 + 8 [pdflush]
 8,0  0  4182 3925.272349191 136 G W 524687 + 8 [pdflush]
 8,0  0  4183 3925.272355058 136 I W 524687 + 8 [pdflush]
 8,0  0  4184 3925.272375453 136 D W 63 + 16 [pdflush]
 8,0  0  4185 3925.272462620 136 D W 524351 + 16 [pdflush]
 
[...] repeats for a while, still on the 3925 second...

 8,0  0  4219 3925.273443527  0 C W 63 + 16 [0]
 8,0  0  4220 3925.273556117  0 D W 524527 + 8 [swapper]
 8,0  0  4221 3925.273989717  0 C W 524351 + 16 [0]
 8,0  0  4222 3925.274088338  0 D W 524687 + 8 [swapper]
 8,0  0  4223 3925.274355706  0 C W 524527 + 8 [0]
 8,0  0  4224 3925.274463268  0 D W 569983 + 8 [swapper]
 8,0  0  4225 3925.274687611  0 C W 524687 + 8 [0]
 8,0  0  4226 3925.274787350  0 D W 571231 + 48 [swapper]
 8,0  0  4227 3925.274848534 136 A W 524687 + 8 <- (8,1) 524624
 8,0  0  4228 3925.274853284 136 Q W 524687 + 8 [pdflush]
 8,0  0  4229 3925.274866415 136 G W 524687 + 8 [pdflush]
 8,0  0  4230 3925.274875355 136 I W 524687 + 8 [pdflush]
 8,0  0  4231 3925.274896029 136 A W 524927 + 8 <- (8,1) 524864
 8,0  0  4232 3925.274899661 136 Q W 524927 + 8 [pdflush]
 8,0  0  4233 3925.274909160 136 G W 524927 + 8 [pdflush]
 8,0  0  4234 3925.274915865 136 I W 524927 + 8 [pdflush]
 8,0  0  4235 3925.274933746 136 A W 524935 + 8 <- (8,1) 524872
 8,0  0  4236 3925.274937378 136 Q W 524935 + 8 [pdflush]
 8,0  0  4237 3925.274946038 136 M W 524935 + 8 [pdflush]
 8,0  0  4238 3925.274961125 136 A W 526327 + 8 <- (8,1) 526264
 8,0  0  4239 3925.274965316 136 Q W 526327 + 8 [pdflush]
 8,0  0  4240 3925.274974535 136 G W 526327 + 8 [pdflush]
 8,0  0  4241 3925.274980682 136 I W 526327 + 8 [pdflush]
 8,0  0  4242 3925.274992136 136 U N [pdflush] 11
 8,0  0  4243 3925.275062820  0 C W 569983 + 8 [0]
 8,0  0  4244 3925.275159486  0 D W 641327 + 8 [swapper]
 8,0  0  4245 3925.276349370  0 C W 571231 + 48 [0]
 8,0  0  4246 3925.276450226  0 D W 697743 + 8 [swapper]
 8,0  0  4247 3925.276714241  0 C W 641327 + 8 [0]
 8,0  0  4248 3925.276806437  0 D W 783799 + 8 [swapper]
 8,0  0  4249 3925.277046146  0 C W 697743 + 8 [0]
 8,0  0  4250 3925.277138621  0 D W 1508311 + 8 [swapper]
 8,0  0  4251 3925.277373581  0 C W 783799 + 8 [0]
 8,0  0  4252 3925.277465777  0 D W 3408143 + 8 [swapper]
 8,0  0  4253 3925.277704089  0 C W 1508311 + 8 [0]
 8,0  0  4254 3925.277800475  0 D W 3604815 + 8 [swapper]
 8,0  0  4255 3925.278067005  0 C W 3408143 + 8 [0]
 8,0  0  4256 3925.278175963  0 D W 524687 + 8 [swapper]
 8,0  0  4257 3925.278249441 136 U N [pdflush] 4
 8,0  0  4258 3925.278436067  0 C W 3604815 + 8 [0]
 8,0  0  4259 3925.278529381  0 D W 524927 + 16 [swapper]
 8,0  0  4260 3925.278806247  0 C W 524687 + 8 [0]
 8,0  0  4261 3925.278906266  0 D W 526327 + 8 [swapper]
 8,0  0  4262 3925.278952922 136 U N [pdflush] 2
 8,0  0  4263 3925.279360819  0 C W 524927 + 16 [0]
 8,0  0  4264 3925.279499392 136 U N [pdflush] 1
 8,0  0  4265 3925.279737145  0 C W 526327 + 8 [0]

So, first syslogd does something at the 3915 second (why to sda and not to ram0?), completing the write after 5 seconds (at 3920); then another 5 seconds later pdflush commits some changes (why 5 seconds? I set it to 5 hours!), and swapper does something too.

The system was also woken once, by apt-get and mlocate. That might be a daily cron thing, I still haven't checked, but I don't mind it running once a day. I plan on waking the drive everyday anyway, to do some "house-keeping", checks on hard drives, syncing of data, and so on.
An hourly wake-up, on the other hand, adds at least 24 startups-spindowns/day to the drive, and that's not healthy! I would really like to stop those hourly wake-ups. Or at least, make them happen at most twice a day or so... Any ideas?

Thanks again
Cristian

EDIT: Maybe I know why pdflush writes. I enabled "laptop mode", so that everytime the drive wakes for some reason, pdflush takes the opportunity to flush its cache, to save another startup when its cache would otherwise expire. If this is true, then the only "culprit" would be syslogd initiating a write to /dev/sda...

EDIT 2: I checked /etc/cron.hourly. It's empty. There is something on /etc/cron.daily, and some of those things I might remove, but cron.hourly is empty.

EDIT 3: Sorry for all these edits but I'm constantly finding new things. I saw that on /etc/crontab there's an hourly entry. It said

17 *  * * *  root  cd / && run-parts --report /etc/cron.hourly

What does run-parts do? Could that be the culprit? I changed that to happen daily, just before the other daily task. I'll leave it running for another 2-3 hours now, and see if anything changed...

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

you could use the lsof to list "open files".
everything being a file in unix, this command is pretty powerful.

say you wanted to know who keeps what files open in a given directory (e.g. /var/log).
you need to be root and enter:

lsof +D /var/log

or say (and this is exactly what we want) you need to know
what files are currently being kept open by a process (syslogd, in this case).
you still need to be root.
and you need to find out your syslogd process id with ps.
then simply enter:

lsof -p <pid>

to find out that the bastard also writes to /dev/xconsole

---

run-parts simply runs scripts or programs in a directory (cfr man page).
/etc/cron.hourly is empty (at least here).
apparently, then, it doesn't do anything.

---

you're defenitely no regular kubuntu user.
but this is fun.
btw, if i were to do what you're doing, i'd install ubuntu server.
then add the minimal kde stuff i need (i.e xserver, basic xfonts, kdm, konsole).
smaller foot print.
server tuning in the kernel.
...

cheers

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

I tried lsof on syslogs, here's the output:

cristian@server:~$ sudo lsof -p 4732
COMMAND PID  USER  FD  TYPE   DEVICE  SIZE  NODE NAME
syslogd 4732 syslog cwd  DIR    8,1  4096   2 /
syslogd 4732 syslog rtd  DIR    8,1  4096   2 /
syslogd 4732 syslog txt  REG    8,1  32080 111704 /sbin/syslogd
syslogd 4732 syslog mem  REG    8,1  38412 80936 /lib/tls/i686/cmov/libnss_files-2.7.so
syslogd 4732 syslog mem  REG    8,1  34352 80940 /lib/tls/i686/cmov/libnss_nis-2.7.so
syslogd 4732 syslog mem  REG    8,1  83708 80930 /lib/tls/i686/cmov/libnsl-2.7.so
syslogd 4732 syslog mem  REG    8,1  30436 80932 /lib/tls/i686/cmov/libnss_compat-2.7.so
syslogd 4732 syslog mem  REG    8,1 1364388 80919 /lib/tls/i686/cmov/libc-2.7.so
syslogd 4732 syslog mem  REG    8,1 109152 47828 /lib/ld-2.7.so
syslogd 4732 syslog  0u unix 0xd5f358c0     12916 /dev/log
syslogd 4732 syslog  2w  REG    1,0 126175   19 /var/log/auth.log
syslogd 4732 syslog  3w  REG    1,0  1465   57 /var/log/syslog
syslogd 4732 syslog  4w  REG    1,0  31526   25 /var/log/daemon.log
syslogd 4732 syslog  5w  REG    1,0 226197   43 /var/log/kern.log
syslogd 4732 syslog  6w  REG    1,0    0   47 /var/log/lpr.log
syslogd 4732 syslog  7w  REG    1,0    0   50 /var/log/mail.log
syslogd 4732 syslog  8w  REG    1,0   301   62 /var/log/user.log
syslogd 4732 syslog  9w  REG    1,0    0   49 /var/log/mail.info
syslogd 4732 syslog  10w  REG    1,0    0   51 /var/log/mail.warn
syslogd 4732 syslog  11w  REG    1,0    0   48 /var/log/mail.err
syslogd 4732 syslog  12w  REG    1,0  37900   28 /var/log/debug
syslogd 4732 syslog  13w  REG    1,0 191304   52 /var/log/messages
syslogd 4732 syslog  14u FIFO    0,14     12855 /dev/xconsole

I suppose the .so files are just read at startup and no more, right? Because there's no way I can put those on RAM. Then, there's /dev/log and /dev/xconsole, but I don't know what they are!

I don't know about cron, maybe that just the "cd /" or the "--report" option were waking the HD? Because it seems that the hourly wakeup is gone. I'm not sure though. Now I'm going to work, and I'll leave the server on, to check how often it wakes in these hours.

About Ubuntu server... I hadn't thought about that!!! I just happened to already have the Kubuntu CD, and installed that... Do you think I should reinstall?
Consider that today I received a new mini-ITX all-in-one motherboard, that - besides being much smaller - is supposed to be totally silent and very power-efficient, so I plan to use that. Could I just plop the system drive there, and let Linux figure out the new hardware? Or would I have to reinstall anyway, as it's the case with Windows?

Cristian

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

Update: I let it run from 10am. Now it's 11:30pm. The btrace log seems to indicate that the hd only woke at 3pm (5 hours after going to sleep, and 5 hours is the time I set in pdflush). So everything seems to be looking good (but why didn't it wake at 8pm? Because nothing got written to cache, so pdflush had nothing to flush?). I might try to increase that time to 12 or 24 hours. After all that will be a file server; it won't use any fancy/new applications, and it'll run lean, no data processing, it won't have many chances to crash. And even if it crashes, there's no actual "unsaved work" that can get lost. If it crashes while I'm in the middle of a file transfer (upload to server) I'll just have to re-upload after reboot...

I'll leave it running some more, and then I'll test if - with these aggressive power-saving settings - the regular server functionality is still OK. And then I'll proceed with the next steps of customization!

Cheers
Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

I suppose the .so files are just read at startup and no more, right?
Then, there's /dev/log and /dev/xconsole, but I don't know what they are!

1) correct
2) /dev/log is a unix socket, /dev/xconsole is a named pipe
ttbomk, /dev/log is where other hosts' sys log messages are read from (cfr syslog man page)
as for /dev/xconsole, i think it's no harm disabling it from syslog.conf
i found at least a post on the net complaining about unnecessary disks wake ups caused by that.

About Ubuntu server... I hadn't thought about that!!!
I just happened to already have the Kubuntu CD, and installed that...
Do you think I should reinstall?
Consider that today I received a new mini-ITX all-in-one motherboard...
Could I just plop the system drive there, and let Linux figure out the new hardware?

1) you're setting up a server, so my kiss rule says: install server. but you don't have to.
2) installation is hw depedendent. a new install is in order here.

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

but why didn't it wake at 8pm?
Because nothing got written to cache, so pdflush had nothing to flush?

that would be an explanation, yeah.
it would also suggest this operating system is pretty well written.
you could schedule a 1 file upload in a given 5 hour window and see how the system reacts.
then pic a different 5 hour window and have your system do (supposedly) nothing.

while we're at it, also notice that not all services log through syslog.
apache, for example, doesn't.
and you'd have to set it up especially.

this is only to say that you shouldn't take that for granted as you add new services...

I might try to increase that time to 12 or 24 hours. After all that will be a file server; it won't use any fancy/new applications, and it'll run lean, no data processing, it won't have many chances to crash. And even if it crashes, there's no actual "unsaved work" that can get lost. If it crashes while I'm in the middle of a file transfer (upload to server) I'll just have to re-upload after reboot...

i guess it's alright.
linux is stable enough.
if your power line is reliable, then, no problems.

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

As for my power line, I'll use an UPS, and the whole server will only use about 20W of power when idle and 50-60 when transferring files, so the battery should last for quite some time. (by the way, do you know if Linux can initiate a safe shutdown when the battery runs low? Does Kubuntu have the necessary drivers and sensors?)

I found that one of the last scripts to be executed when entering runlevel2 is /etc/rc.local. So I added these lines to it:

echo 5 > /proc/sys/vm/laptop_mode

mke2fs /dev/ram0
mount /dev/ram0 /var/log

if [ -x /home/cristian/logs ]; then
 rm -r /home/cristian/logs
fi

mkdir /home/cristian/logs
mount /dev/sdb /home/cristian/logs
rsync -rv /home/cristian/logs/log /var/log

hdparm -S24 -B1 -A1 /dev/sda

echo 8640000 > /proc/sys/vm/dirty_expire_centisecs

exit 0

It gets executed. The "if" branch doesn't look very elegant to me, I wanted to so something like "If the folder does not exist, THEN create it" instead of "If the folder exists THEN delete it; END IF; then re-create it" but I don't know how to program scripts....

Now I have to test the "file serving" functionality, then add a few cron tasks... The fun has just begun!

Cristian

[noelvh]

Re: Hard drive spindown: root drive wakes frequently

Here is some thing that might fit your needs.
freenas.org this is a free nas that has power management added to the OS.
I have a box running it but have not got it fully configured. It only takes about 15 min to install and setup.

Keep in mind once you boot from the CD you will need to hit the CTRL+ALT keys to get the install menu.
Then you can log in using a web interface and do all the config work in a gui fashion.

I know this is not the path you were taking but might be one to look at.

Noel Vh

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

I already tried freenas (and another couple of similar solutions). But I had to abandon it for two reasons. The main reason is that it won't allow more than one share per partition, while I need to have a single partition with several shares (to better use disk space). The second reason is that I like to have a "standard" Linux distro that I can tweak and customize to my needs! With freenas and other similar solutions, you are pretty much stuck with what you have, you can only customize it as much as the menu will allow you...

Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

mate, here's an init script template.
once we're done with it, you can place it in /etc/init.d.
i'll take a look at what you post ealier and fill this is...
for now just save it away somewhere...

#!/bin/sh -e

PATH="/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin"

# load default init functions
. /lib/lsb/init-functions

case "$1" in
start)
log_action_begin_msg "Configuring my stuff"
# STARTUP STUFF GOES HERE
# ...
;;

stop)
log_action_begin_msg "Deconfiguring my stuff"
# SHUTDOWN STUFF GOES HERE
# ...
;;

*)
echo "Usage: /etc/init.d/foobar {start|stop}"
exit 1
;;
esac

exit 0

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

alrigth...
the attached script starts looking more like it.
it called .txt 'cause it wouldn't allow me to attach it otherwise.
but .txt shouldn't be there.
you can test it manually, to make sure it do what's needed...

/etc/init.d/foobar start

/etc/init.d/foobar stop

/etc/init.d/foobar wrongoption

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

Hmmm... I created a file named foobar in /etc/init.d, gave it permissions 0755, I put inside it the script you gave me.
If I run

/etc/init.d/foobar start

it says "Configuring my stuff... Fail".
Same thing if I run "foobar stop". It doesn't say what part fails though.

Anyway, if it gets working, what should I do... add the line

/bin/sh /etc/init.d/foobar start

to the rc.local? Or create something like a "S99foobar" link in /etc/rc2.d?

Another thing I was looking into... My laptop can scale the CPU frequency automatically. Is there a way to set that up from the command line, so that when the system is idle the CPU freq goes down?

Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

my script is not supposed to fail...
you ran it as "root", yeah?
i tested it here and seemed to be working alright.
you can turn debugging on with "-c" instead of "-e".
that's on the first line of the script, that is...

Anyway, if it gets working, what should I do...
add the line

/bin/sh /etc/init.d/foobar start

to the rc.local?
Or create something like a "S99foobar" link in /etc/rc2.d?

number 2.
you can use "update-rc.d" for that.
it will automate the creation/removal of the links.
you only need to tell it at what levels you want to start/stop your service.
and at what point in the sequence.

this might be and idea:

update-rc.d foobar start 01 2 3 4 5 .
update-rc.d foobar stop 99 0 1 6 .

it'll set things up before anything else (S01) at levels 2, 3, 4, and 5.
and will stop thing after everything else (K99) at levels 0, 1, and 6.

Another thing I was looking into... My laptop can scale the CPU frequency automatically. Is there a way to set that up from the command line, so that when the system is idle the CPU freq goes down?

mmm...see here:

cat /sys/devices/system/cpu/cpu<cpu#>/cpufreq/scaling_governor

i think all you need is to "echo >" the policy you want in the corresponding cpu# file.
i have a dual core, so i have .../cpu/cpu0/... and .../cpu/cpu1/...
ttbomk, accepted values are:
- performance
- powersave
- userspace
- ondemand
- conservative

guess "ondemand" is what you need.

i love linux

cheers

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

my script is not supposed to fail...
you ran it as "root", yeah?

Sorry... It still fails! I ran it as root, yes.
I added it to /etc/init.d and when I run

sudo /etc/init.d/foobar start

I get immediately

 * Configuring my stuff...                                [fail]

If I change the -e to -c, the console hangs for a looong time, and then at last says

/bin/sh: Cannot fork

update-rc.d foobar start 01 2 3 4 5 .
update-rc.d foobar stop 99 0 1 6 .

I tried that. Well, I changed the "start" number to 99 and the "stop" number to 01, because I like the script to be the last thing to run at boot, and the first at shutdown).
The first command worked (now I have S99foobar link in the rc2.d and other folders). But the second command fails with this error:

cristian@server:/etc$ sudo update-rc.d foobar stop 01 0 1 6 .
System startup links for /etc/init.d/foobar already exist.

mmm...see here:

cat /sys/devices/system/cpu/cpu<cpu#>/cpufreq/scaling_governor

No luck. Inside my /sys/devices/system/cpu/cpu0 folder, I don't have a /cpufreq folder. And this is both on the server and on my desktop.

I also found that I need to manually set my smb4k configuration to use "utf8 charset" if I want accented characters to be consistent with those on the server.

Cristian

[EDIT]

Damn. As soon as things start to look good there's always new (bigger) problems popping out.
Until now I have been experimenting with only the "system" drive connected. I had made a partition where I created the shares. I thought "Let's configure the system with one drive for now - then I'll add the data drives".
I thought that would be trivial - after all they're *data* drives, they're only accessed when someone on the network wants the data.

So today I connected them. I configured them, made the shares, transferred the data. All was looking good. Fast transfers, good performance. I was happy.
Then I tried, to test the spindown, the command

sudo hdparm -S5 /dev/sdc

which should put to sleep the drive after 25 seconds of non-use. I also ran btrace to check that nothing was accessing the drive.

Well, nothing accesses it, but it doesn't spin down. >
If I give the command

hdparm -Y /dev/sdc

or

hdparm -y /dev/sdc

the drive spins down regularly. So why not with the -S option?
These drives are modern, 500GB Western Digital SATA drives. They were installed in a small NAS box, and they used to spin down regularly there. So why not on my new server?!? Aaaargh!!!

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

sorry.
my fault.
shell debugging is "-x" not "-c".
typo.
could you please run the script again and see what it says?

as for the rest...

don't know why the drive doesn't spin down.
though, you new drive should be sdb and the usb pen should now be sdc...

you'll have to google to see how to enable "cpu frequency scaling".
it has to do with acpi and kernel.
i have this:

root@crisps:~# find /lib/modules/2.6.22-14-generic/ -name \*cpufreq\*
/lib/modules/2.6.22-14-generic/kernel/drivers/cpufreq
/lib/modules/2.6.22-14-generic/kernel/drivers/cpufreq/cpufreq_powersave.ko
/lib/modules/2.6.22-14-generic/kernel/drivers/cpufreq/cpufreq_conservative.ko
/lib/modules/2.6.22-14-generic/kernel/drivers/cpufreq/cpufreq_stats.ko
/lib/modules/2.6.22-14-generic/kernel/drivers/cpufreq/cpufreq_ondemand.ko
/lib/modules/2.6.22-14-generic/kernel/drivers/cpufreq/cpufreq_userspace.ko
/lib/modules/2.6.22-14-generic/kernel/arch/i386/kernel/cpu/cpufreq
/lib/modules/2.6.22-14-generic/kernel/arch/i386/kernel/cpu/cpufreq/cpufreq-nforce2.ko
/lib/modules/2.6.22-14-generic/kernel/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.ko

and this:

root@crisps:~# lsmod|grep cpufreq
acpi_cpufreq      10568 1
cpufreq_userspace    5280 0
cpufreq_ondemand    9612 0
cpufreq_stats      7232 0
cpufreq_conservative   8072 0
freq_table       5792 3 acpi_cpufreq,cpufreq_ondemand,cpufreq_stats
cpufreq_powersave    2688 1
processor       32072 2 acpi_cpufreq,thermal