[SOLVED] Hard drive spindown: root drive wakes frequently

[Quaxo76]

Hello all,
I've been setting up a bare-bone file server for my home network (see post in the "networking" section). The file transfer is more or less working now, so I need to start tuning the server to suit my needs.
I need it to be as silent and as little power-hungry as possible; so I would like to spindown the hard drives after a few minutes that they're not used.

I've tried the command

sudo hdparm -S24 /dev/sda

which shuts down the drive after 2 minutes of inactivity (after all, the server might sit idle for hours, or days at a time, without being accessed).
The command works - the drive spins down, but then it starts again a few minutes later, thus cycling on-off several times per hour. That's bad for the drive's health.

Using atop I saw that the process "syslogd" wants to write something to the HD; the write is cached, and then about 30 seconds later the write is committed to disc by "pdflush" - and the drive is woken.

So I would think there are 2 possible ways to stop this behaviour:
1) prevent syslogd from trying to write so often.
2) increase the "expiration time" of pdflush, so that it flushes to disk, say, once a day and not more (it's OK if the drive spins once a day, because I plan to setup a nightly rsync job anyway).

Any ideas on pros and cons of the 2 ideas above? Any other ideas?
And, how would I actually modify the system to implement those solutions, as I couldn't find any specific info around?

Thanks
Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

the proper way to go about this, i think, would be
to have /var/log (or the whole /var) on a usb pen drive.

syslog collects messages from all over the system
and you would need a complex tuning to do what you want.

and i wouldn't even take into consideration
setting the system up to sync to disc once a day.

m2cts.

hth.

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

That's an interesting idea. But what about the flash chip "wear"? Flash memories are rated for a limited number of writes, and wouldn't writing on a pendrive every 5 seconds bring it to a premature death?
Maybe I could set it up to write every 10 minutes or so, would that still be too much?

Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

true.
yeah.
you're right.
but...
given the average size of /var/log (mine is constantly around 7.5mb)
and the cost of a usb pen, it might be worth running a real world test
and see how long it actually lasts...

i can't see any other solutions to your problem.

unless, of course, you wanted to disable syslogging all together.
it's not that people look at logs much anyway...

cheers

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

Would it be OK for the system to have /var/log on a usb key? Should I just add the pendrive on /etc/fstab and mount it on /var/log? Would this be transparent to the system?
Also, if I wanted to configure (or disable ) syslogging, how would I do that? Is there a config file to edit?

Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

Would it be OK for the system to have /var/log on a usb key?
Should I just add the pendrive on /etc/fstab and mount it on /var/log?
Would this be transparent to the system?

totally transparent, yes.
to the os, it'll just be a directory like any other.

Also, if I wanted to configure (or disable ) syslogging, how would I do that?
Is there a config file to edit?

ttbomk, the only thing you can configure is what gets logged.
that's in /etc/syslog.conf (there's a man page in section 5 for it).
wanting to disable logging...
it should be a matter of avoiding that klogd and syslogd start at boot up.
so i guess that moving the following rc init script links out of the way (and reboot)

root@crisps:~$ find /etc/rc?.d -name \*logd\*
/etc/rc1.d/K89klogd
/etc/rc1.d/K90sysklogd
/etc/rc2.d/S11klogd
/etc/rc2.d/S10sysklogd
/etc/rc3.d/S11klogd
/etc/rc3.d/S10sysklogd
/etc/rc4.d/S11klogd
/etc/rc4.d/S10sysklogd
/etc/rc5.d/S11klogd
/etc/rc5.d/S10sysklogd

should be suffcient to do the trick.

the start up procedure looks for S file names to start services.
the shut down procedure looks for K file names to kill services.

so, you just need to add whatever prefix to those file names
to have your system simply ignore them.

renaming them that way, rather then removing them,
will make it easier to restore the original configuration
should you want/need to.

i'd have a crack at the usb pen thing first, though...

cheers

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

I just found an old 128Mb pendrive. Tonight or tomorrow I will experiment with that... I suppose 128 Mb is plenty of log space!
In case the pendrive blows up, wiping the logs folder, what happens? Can I just plug a new one? Will the system complain for the lost files?

And, last question (for now): I have disabled startup of X, to keep the system as "lean" as possible (it has little RAM, and I run it without fan on the CPU, so I want it to be lean). How do I mount the pendrive? Does it always get the same dev name at every reboot (i.e. /dev/sdc1 or something)? How do I find its /dev name without starting X? Or should I mount it "by name" or "by UUID"?

Thanks
Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

I just found an old 128Mb pendrive.
Tonight or tomorrow I will experiment with that...
I suppose 128 Mb is plenty of log space!

it should be plenty, yeah.

In case the pendrive blows up, wiping the logs folder, what happens?
Can I just plug a new one? Will the system complain for the lost files?

i'd just mount it on top of the current /var/log.
if it blows up, you'll still have the old one underneath.
in any case, you can safely rm -rf everything you have in /var/log.
as they start up, services will create new log file if they don't find old ones to append to.

And, last question (for now): I have disabled startup of X, to keep the system as "lean" as possible (it has little RAM, and I run it without fan on the CPU, so I want it to be lean).

good idea.

How do I mount the pendrive?
Does it always get the same dev name at every reboot (i.e. /dev/sdc1 or something)?
How do I find its /dev name without starting X?
Or should I mount it "by name" or "by UUID"?

1) by adding an entry in /etc/fstab
2) if you don't add any new devices, it always gets the same dev name, yeah
3) by entering "sudo fdisk -l"
4) by uuid is likely the best choice (use the "sudo blkid" command to find out what's what)

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

Thanks for the very detailed reply. I will try that when I come back from work.
As another idea, would it be possible to mount /var/log to ram instead of flash drive, and "flush" that to the pendrive every hour or so? Which would dramatically increase flash life, while keeping a hard copy of almost all of the logs... Would this be a good idea? Is this even feasible?

I'll report after trying the pendrive thing.

PS - I noticed that another process that writes to the drive is "nmbd". That seems to be a net bios resolver or something like that, and it's associated with samba. With that machine being a file server, of course I can't disable it... But what is it writing? And where? Maybe I should mount that to pendrive too? Though I haven't checked *how often* it writes to disk, it might be seldom enough to be acceptable...

Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

Thanks for the very detailed reply. I will try that when I come back from work.
As another idea, would it be possible to mount /var/log to ram instead of flash drive, and "flush" that to the pendrive every hour or so? Which would dramatically increase flash life, while keeping a hard copy of almost all of the logs... Would this be a good idea? Is this even feasible?

yes.
ram disk is your keywords here.
build an ext3 file system on a ram disk and mount it to /var/log.
you can then back up /var/log to the pen drive every so often.
ubuntu creates a few default ram disks.
issue:

ls -l /dev/ram*

to see how many you have (i have 16)
and:

dmesg|grep RAMDISK

to check how big each default ram disk is (mine are 64mb each)

PS - I noticed that another process that writes to the drive is "nmbd". That seems to be a net bios resolver or something like that, and it's associated with samba. With that machine being a file server, of course I can't disable it... But what is it writing? And where? Maybe I should mount that to pendrive too? Though I haven't checked *how often* it writes to disk, it might be seldom enough to be acceptable...

guess you'll find other stuff that writes to disk...
as for samba...
i don't know much about it.
i don't interoperate with windoze.
i've managed to avoid having to do with windoze so far.
for sure you can remove samba if you don't have windoze machines about.

cheers

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

Jankushka,
I tried the pendrive thing. It seems to be working. It was rather easy - it's /dev/sdb, I just had to give it a mount command!
Now, the hard drive seems to stay "spinned down" for most of the time. But I can't "baby-sit" it for hours, and I would like to know how much it really cycles power. Is there a way to log to screen (or to log file) all the hard disk activity, and possibly all the powerups/powerdowns? I use atop but it only shows who's *currently* accessing the drive, while I'd like to have an "history". That way I'd just let it sit there for a day or so and then check the log to see who/when accesses the disk...

The ramdisk thing didn't work for me. I fdisk'd /dev/ram0, created a partition, but when I saved and committed the changes, it said changes were committed but would be used by the kernel at next reboot. And I obviously couldn't mount it.

Cristian

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

Now, the hard drive seems to stay "spinned down" for most of the time. But I can't "baby-sit" it for hours, and I would like to know how much it really cycles power. Is there a way to log to screen (or to log file) all the hard disk activity, and possibly all the powerups/powerdowns? I use atop but it only shows who's *currently* accessing the drive, while I'd like to have an "history". That way I'd just let it sit there for a day or so and then check the log to see who/when accesses the disk...

check out sar.

The ramdisk thing didn't work for me. I fdisk'd /dev/ram0, created a partition, but when I saved and committed the changes, it said changes were committed but would be used by the kernel at next reboot. And I obviously couldn't mount it.

you don't have to partition it.
simply make an fs on it.

cheers

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

Thank you very much! It worked. I wonder why I thought it had to be partitioned.
Now the /var/log is in RAM.
Now, if I want to automate the thing, I thought about something like this:

At every boot:
- mount /dev/sdb /logs_backup (/dev/sdb is the USB pendrive)
- mount /dev/ram0 /var/log
- rsync /logs_backup -> /var/log

Every 6 hours or so, and at shutdown/reboot:
- rsync /var/log -> /logs_backup

Would this be correct? And how can I tell the system to execute something at startup and at shutdown? (for the "every 6 hours" thing I should use cron, right?)

Cristian

EDIT - Further question: I noticed pdflush is the most frequent writer to hdd. I want to increase the delay before cached data expires and is flushed to disk. I suppose that at shutdown/reboot the system automatically flushes the cache to disk, right? (I mean a normal shutdown/reboot, not a crash)...

[jankushka]

Re: Hard drive spindown: root drive wakes frequently

Now, if I want to automate the thing, I thought about something like this:

At every boot:
- mount /dev/sdb /logs_backup (/dev/sdb is the USB pendrive)
- mount /dev/ram0 /var/log
- rsync /logs_backup -> /var/log
...
Would this be correct?

yeah, just remember to rebuild the fs on the ram disc before mounting it...

And how can I tell the system to execute something at startup and at shutdown?
for the "every 6 hours" thing I should use cron, right?

1) rc?.d directories contain links to /etc/init.d scripts.
these scripts are all based on the same start/stop/... template.
they are linked from /etc/rc?.d.
links starting with S mean start (i.e. will invoke the init.d script with the "start" parameter).
links starting with K mean kill (i.e. will invoke the init.d script with the "stop" parameter).
the number defines the order in which links are read -> scripts are executed.
the ? in the rc?.d corresponds to the operating system run level.
update-rc.d is the command you use to place the S or K links in the different run level dirs.
that's once you've written your init.d script.

2) right, yes.

EDIT - Further question: I noticed pdflush is the most frequent writer to hdd. I want to increase the delay before cached data expires and is flushed to disk. I suppose that at shutdown/reboot the system automatically flushes the cache to disk, right? (I mean a normal shutdown/reboot, not a crash)...

filesystems are defenitely sync'ed when they're unmounted.
filesystems get unmounted during (at the end of) a regular shutdown procedure.
so, yes, your statement is correct.
but i'm not sure i'd venture into tweaking pdflush (or the kernel).
you could mount your filesystems with "noatime" and "nodiratime".
that would further reduce the need of updating the fs.

next?

[Quaxo76]

Re: Hard drive spindown: root drive wakes frequently

yeah, just remember to rebuild the fs on the ram disc before mounting it...

Sure... I just forgot to add that!

1) rc?.d directories contain links to /etc/init.d scripts.
these scripts are all based on the same start/stop/... template.
they are linked from /etc/rc?.d.
links starting with S mean start (i.e. will invoke the init.d script with the "start" parameter).
links starting with K mean kill (i.e. will invoke the init.d script with the "stop" parameter).
the number defines the order in which links are read -> scripts are executed.
the ? in the rc?.d corresponds to the operating system run level.
update-rc.d is the command you use to place the S or K links in the different run level dirs.
that's once you've written your init.d script.

Hm. More complicated than I thought. I was hoping in something like the good old DOS Autoexec.bat... a simple script where you just "plop in" any commands you want to be run at startup! Looks like I'll have to study some more...

but i'm not sure i'd venture into tweaking pdflush (or the kernel).

Well, I changed the "dirty expiration" value to something much bigger, so that the system doesn't feel the need to flush so often. So far I haven't seen ill-effects. I'll have to see what happens when I start reading/writing from/to the shares!
As for the kernel, I haven't touched it and I'm not planning to!

you could mount your filesystems with "noatime" and "nodiratime".
that would further reduce the need of updating the fs.

I'll try that too asap.

For now, the server system has been running "idle" - i.e. with no action on my part - for about 2.5 hours. /dev/ram0 has only been written to 5 seconds after starting the monitoring tool (I used btrace, it's quick to setup even though it doesn't monitor startup/spindown).
/dev/sda instead, has been written to right after starting the tool (which was expected) and then after exactly one hour. I wonder why the "exactly one hour" interval, looks like something planned... It didn't startup at the start of the second hour though. I think I'll leave it running overnight...

next?

Well, since you've asked... Any idea why - considering that now /var/log is mounted on ram - syslogd is still issuing I/O commands to /dev/sda?

Well... I really appreciate the help you're giving me. I've been using Linux for quite some time, but this is the first time I try something so much "off-standard", and the first time I run a machine who has no X at all, and is serviced remotely via ssh... It's a challenge for me, there's so much to learn! But it's so rewarding when something eventually works as intended!

Cristian