Pre-install setup help, please.

[oshunluvr]

ESP EFI GPT MBR - a lot of confusion around these topics. I've covered most of it elsewhere, but I got a few minutes, so...

MBR is the older drive partitioning system that limited you to 4 Primary partitions but allowed a single Extended partition that could hold some number of Logical partitions.
GPT is the current partitioning scheme and removes the use of Extended and Logical partitions and, when using Linux, allows 128 Primary partitions (it will actually allow more, but that's another very long topic). GPT also has some other great features, like built-in partition table backups, etc.

My understanding is by default, UEFI firmware will not boot to MBR disks because UEFI executes some code in a location that conflicts with MBR headers. Interestingly, GRUB will not install to GPT disks by default either because it's header uses part of the area the GPT partition table uses. In short: GPT and GRUB are somewhat incompatible.

Enter the partition types EF00 and EF02. parted flags these partitions this way:

EF00 = boot, esp
EF02 = bios_grub

So the solution to use GPT with GRUB, or install using UEFI is to create the first partition as either type EF00 for use with UEFI or EF02 for use with GRUB.

For EFI, if you use partition type EF00, the "boot, esp" flags are set automatically, thus avoiding the "Missing ESP" errors some report. You can format it with fat32 and it retains the flags and partition type. This tells EFI it's OK to use this partition.

For GRUB, make an EF02 type partition and don't format it at all. By default now the partitioning tools start the first partition at sector 2048 so that proper alignment (partitions to sector boundaries) occurs. This helps with usable space and supposedly performance. BUT this leaves a whole 1000 KiB unused - the space from sector 34 to 2047. Here's a neat trick: You can use this space for your EF02! It's like free space!. 1000 Kib is plenty of space for GRUB. Just set the sectors manually using gdisk and make your first partition in this free space.

Now EFI is another matter. Lots of misinformation out there, but I can say the minimum size for a fat32 partition on an advanced format drive is 32MB. I doubt that's large enough if you're booting more than two installs. MS says 100MB and the creator of EFI says 550mb. Seems like a lot. My laptop came with a 260MB EFI partition and now with 3 distros and Windows installed it's used 34M. I think you can make it much smaller than 550MB.

[TWPonKubuntu]

@oshunluvr, thanks for the explanation on the various partition and formatting options. It still remains convoluted and confusing for newbies and I see no simply solution. Perhaps a "How-to" document with examples would help? I don't have such a document, nor the expertise to create it. So this is a call-out for more input from our gurus...

[Don B. Cilly]

Yeah, that's perfectly clear, Mickey, yeah. Just give me one minute to confer with my colleague... [Did you understand a single word of what he just said?] --- Jason Statham in "Snatch".

I just had a look at KDE's partition manager. Creating a new partition does not have an option for btrfs.
Let alone EF00, 02 or whatever.
Setting flags, gives you 1 (one) option: Boot.

Gparted, does have btrfs.
Setting flags, ten options.
Still no EF0x, so... what are those?

I mean, when you say:

So the solution to use GPT with GRUB, or install using UEFI is to create the first partition as either type EF00 for use with UEFI or EF02 for use with GRUB.

How would we do that? Using command-line parted? I'm surreptitiously looking one side and the other like "Who, me?"

Also, I seem to be using EFI with GPT and grub here, and am still able to type...
I mean, it's all perfectly clear, but... :·/

[oshunluvr]

I just had a look at KDE's partition manager. Creating a new partition does not have an option for btrfs.

Incorrect. You may not have the tools installed.



also notice the flags at the bottom of the second screen grab. However, I never mentioned using KDE Partition Manager at all, or gparted or gnome disks. or any GUI tool. My KDE partition manager does show the bios-grub flag along with boot, but I wouldn't use a GUI tool to create a new system anyway. GRUB works with EFI but EFI is not mandatory unless you require Windows. If you want GRUB on GPT without EFI you have to make it happen.

I really just intended to bring more into the discussion as I've already written a how-to on GPT and GRUB here and it fully explains how to go about it. EF00 isn't mentioned because I hadn't used EFI at all, but the process would be the same.

GUI tools are all well and good if you're satisfied with what the GUI developer has offered you. However, the GUI is a limitation not a benefit in many ways. One can easily just had the system over to the installer and let it decide what your setup will be like and it will install and work fine. However, some of us want more control over how things are and so ground work is required. The OP asked about what to do before installing so I offered some advanced information.

[jglen490]

I guess we'll just have to disagree on the utility of BTRFS.

As far as backups, be very careful about relying on a backup scheme that only includes snapshots stored on the same drive (or cluster) as the actual data. If the drive goes south, there is no recovery. Always include, or better yet solely use, a backup scheme that uses an external detachable drive as the target. That's a scheme that will satisfy most of the OP's " 'ilities".

Again, for the OP, BTRFS is a choice not a requirement. Do your due dilligence and research speed trade-offs, data security, backup/recovery mechanisms, reliability, and all the rest. BTRFS and ext4 are two completely different animals. If I were the lead SA or DBA for a Linux-based enterprise system and had lots of budget, I would insist on a set of management characteristics that MIGHT result in something like BTRFS but more probably ZFS.

And since I'm not that lead SA/DBA, and my system is small and single user, ext4 is perfect.

[oshunluvr]

As far as backups, be very careful about relying on a backup scheme that only includes snapshots stored on the same drive (or cluster) as the actual data. If the drive goes south, there is no recovery. Always include, or better yet solely use, a backup scheme that uses an external detachable drive as the target. That's a scheme that will satisfy most of the OP's " 'ilities".

Nobody suggested that snapshots were backups. A backup by definition would require it to be on a separate device, regardless of it's format. I simply stated that backups can be made from a BTRFS file system without the need of external utilities like you do with almost every other file system, which is a fact. Snapshots fill an entirely different purpose than a backup, hence my example of using a snapshot to "roll back" rather than re-installing or spending hours restoring from a backup.

[chezler]

Quite the topic, here. Sure glad I came here to get info before just jumping in.
oshunluvr, thanks for the input. I saw a thread , pretty long one , where you and GreyGeek were all over btrfs. That was last night, kept me up late.
After reading that, and your input here, and everything else I've read about it, I have to say; That's the way I'm leaning.
You're right, I'm new and learning. I want to learn more, and I want to learn the best stuff not just the easiest or "way it's always been" ,... btrfs looks like the way to go.
Something in that thread I read last night really HIT; ext4 is no longer adding new features, it's old, and probably on the way out. It is faster, yes, but fast isn't everything. Functionality is better.

More research,... I'll look up your thread on installing, and read more by GreyGeek. Don B. Cilly just said some stuff that makes me hesitant about sticking in that thumb drive just yet.

I just took a peek at the thread from 8/11/17 you mentioned, oshunluvr, don't have time to go over it right now, but will later today. Thanks again.

I have to go do some things and will be back later today to read some more.

Thank you to everyone for all the great info. Lets me believe I chose the right place and the right Distro to get started. You are all being so generous.

Charlie, signing off, later.

[jglen490]

Nobody suggested that snapshots were backups. A backup by definition would require it to be on a separate device, regardless of it's format. I simply stated that backups can be made from a BTRFS file system without the need of external utilities like you do with almost every other file system, which is a fact. Snapshots fill an entirely different purpose than a backup, hence my example of using a snapshot to "roll back" rather than re-installing or spending hours restoring from a backup.

O.K., fair enough. So give me an example of something that would need to be "rolled back".

[Don B. Cilly]

Don B. Cilly just said some stuff that makes me hesitant about sticking in that thumb drive just yet.

Don't mind me, I'm just being cilly. Really.
I don't know anything about boots. I'm a sailor, not a cobbler ;·)

About sticking in the thumb drive, though, I don't see why not.
Re-installing becomes a bit of a bother after weeks of customising, fine-tuning, etc.
Just try it, you don't like the set-up, you can always re-install from scratch.
You do some personal work on it, copy it to an external drive or something.

[chezler]

Don't mind me, I'm just being cilly. Really.
I don't know anything about boots. I'm a sailor, not a cobbler ;·)

About sticking in the thumb drive, though, I don't see why not.
Re-installing becomes a bit of a bother after weeks of customising, fine-tuning, etc.
Just try it, you don't like the set-up, you can always re-install from scratch.
You do some personal work on it, copy it to an external drive or something.

You got a point there Cilly, Back in The Day, We were re-installing Windows all the time. Trying different versions on different machines. And if you had Win Me, you reinstalled and thinned it out every 6 months just so it would run right.

Last night (again, like the old days) up till 2:30 reading about gdisk. I was amazed that it reminded me of setting up hard drives, running fdisk, registry and services tweaks.
I downloaded Rod Smith's fdisk last night so I could have gdisk.

Question,... Is this the same fdisk we used years ago, or something else with the same name?

In order to Run gdisk From Windows; does it need to be installed in Windows, installed to a thumb drive, or extracted to a thumb drive, or does it matter?
Will it run from a thumb drive in Windows?
R Smith was rather vague in those areas. Plus everything I've found on the subject, it's being run from Linux.

It looks simple enough to use, I mean, what's the worst you can do? Miss-type something and wipe your hard drive and have to start over from scratch.
That risk has been there since computers were invented. Never stopped any one before or since. besides, looks to me like that's half the fun of it all.
Why else would we stay up to all hours reading and tweaking.

[oshunluvr]

So I didn't really address your questions directly. Here's my thoughts;

First off; Hello My name is Charlie. New on the forum. Pretty new to Linux.

Welcome to KFN!

I have installed Linux twice in the past just to test the waters, knowing that one day I would drop Windows for good. That day is rapidly approaching since a Windows update borked one of my Windows 10 installs a few weeks ago. I really don't like W10, I mainly use Win7, But its days are numbered.

Computer are like air conditioners - neither work well with windows open! For the thing I absolutely have to have windows for (mostly work related) I use a VM.

I had Open Suse on a dual boot with Win XP years ago. Had Ubuntu on triple boot with Win7 and 10 about a year+ ago. Didn't do much in it either time, but boot regularly to play around. I'm ready to get serious now.
I want Kubuntu 18.04 LTS on my laptop for now, and will eventually do a dual boot with Win7 on my desktop.

I have built many Windows based computers, both from new components and from salvaged components. Also maintained them and updated and upgraded as needed. Somewhere between 50 and 75 total. Also, I worked for 10 years in a production facility as Asst. Quality Mgr. where, as part of my responsibilities, I had to maintain and upgrade 20 production floor computers as well as teach people how to use them and enter data.
I have been using since 1985, and building since Windows 98.

We have a similar history.

...CMD...

LOL, dead giveaway that you've been using Windows! The short hand you'll see here for command line or terminal entry is usually "CLI" for "command line interface". We use Konsole or if you use the CLI a lot like I do, I use Yakauke which is a drop-down terminal. A simple F12 and it drops open from the top of the screen and closes automatically when I'm done with it.

Sounds like you've got plenty of fast SSD storage. Be cautious about what advise you take off the web about SSD management. SSDs have rapidly improved and most distros have kept up with them - meaning the defaults are usually fine. Anything you read more than a few months old is likely out of date.

Partitions. Need a UEFI of ? size. 512 MB, 1GB ? What I've read makes it kinda vague. The best answer I've found is to make as big as you need.

I answered this above, but I think 100MB is plenty if you go with UEFI to boot unless you're planning on 10 or more distros.

Does Kubuntu have EXT4 as default? Can't find that answer. I've read that it uses or can use ZFS. I like the Idea of that file system if it can be used reliably.
If EXT4, do I need a swap partition?

I addressed this too, but EXT4 is the default file system that the installer (Ubiquity) uses, but there's no real "default" file system. You can pick whatever you want. ZFS has been reported on rather well here - search for GreyGeek's posts on it. It's not in the kernel so not supported by any distro so you're on your own with that one.

If you don't have a swap partition, Ubiquity will create a swapfile instead. You can change this later if desired. IMO if you're going to dual boot any other Linux installs, even just a backup install, you should use a swap partition. The reason being that multiple installs can share a swap partition, but each one of them will need a swapfile if you go that route. The math is easy here: one 16GB swap partition or Xdistros times 16GB swapfiles. Note that if you go with BTRFS you must use a swap partition. COW (copy on write) file systems do not handle dynamically sized files like swap files or dynamically sized virtual drives.

BTW, IMO having your swap partition on the fastest device makes sense. You bought that pricey stick of silicon to speed things up and swapping to RAM is the slowest thing your computer does. Many folks worry about excess wear but I probably hit swap a couple times a year, so where's the wear in that? If you hibernate or suspend, you'll use swap more often but still much less than you'll use the data space on the same device.

What size?
Do I need any other partitions? I've seen examples of systems with 1 partition using the whole drive, and I've seen others with several.

Swap is generally sized to match the size of your RAM. This is to allow hibernate or suspend.

As far as other partitions, part of the BTRFS discussion above I touched on it. Here's what I do and I use: I have 8 or 9 VMs at any given time and 4 to 8 distros installed (I like to play a bit!). All my Linux installs are on a single BTRFS file system along with a stand alone GRUB install and home (all in separate subvolumes). I boot using GRUB and use GPT partitioning so I need a GRUB_BIOS partition. I have a swap partition. Because I use a lot of VMs I use dynamically sized virtual drives so I have an EXT4 partition for those. I have a 512GB nvme SSD drive as my main boot drive, so it has my three main partitions: GRUB_BIOS (1MB), SWAP(16GB), and the main BTRFS partition (460GB). My EXT4 partition for the VMs is on a hard drive and 500GB. My laptop is set up similarly but it has a single 1TB nvme drive so everything is on it. If you go with EXT4 and plan on doing other installs, you'll need at least one more partition for home (not mandatory but advised) and several more to install other distros to.

I believe I should be able to just boot to the thumb drive, and the installer will do what it thinks is best. (For most people).
I want what is best, fastest, but mainly most stable.
I am here for the long haul, not just playing around. I want Windows off my computers.

As I said before, there's nothing wrong with letting the installer do it's thing. It won't be bad. However, if you're ready to roll up your sleeves and get to learnin' you could start with customizing your install right off the bat. Most of us start vanilla then as we expand our knowledge do more and more. You'll probably re-do your installation a few times before you get it just right for you. Not a big deal - I can do a Linux install in less time than it takes Windows to do an update. You might even end up with a different distro you like better than Kubuntu, but you'll still be welcome here. Again, welcome to KFN. You'll find lots of helpful people here and we try to enjoy civil discourse. I look forward to your participation.

[oshunluvr]

O.K., fair enough. So give me an example of something that would need to be "rolled back".

The most obvious example is video driver installation. I think everyone who uses nvidia or amd video drivers has had at least one bad installation leaving them with no desktop. Simply take a snapshot, upgrade or install the video driver, and reboot. If it fails, change to the snapshot and reboot.

I posted here in the KDEneon forum last week: My KDEneon install and home subvolumes are on an automatic daily snapshot via cron. Once a week the oldest snapshots (7 days old) is transmitted using BTRFS send|receive to a backup drive in my system. So I have a running set of 7 daily snapshots and a weekly backup. This seems sufficient for my needs.

Last Wednesday I ran update and several dozen packages were updated and a new kernel installed, so I rebooted. I hadn't rebooted in several days. The boot failed before SDDM loaded - black screen, no keyboard access, no drive activity, nothing. Since I have several bootable distros on my system, I rebooted and selected Kubuntu. Once I had logged in, I navigated to the root BTRFS file system (above the subvolume level) and I renamed @KDEneon (the subvolume I use) to @KDEneon_bad. I then snapshoted the Tuesday snapshot (to duplicate it) as @KDEneon and rebooted. This also failed to boot. I repeated the above steps and this time used the Monday snapshot, and it booted.

So whatever I had done between Monday and Wednesday had caused the system to not boot. I then re-did the update and I'm back up without issue. I don't know if an update failed or a bad package or what caused it, I never figured it out. I then deleted the three "bad" snapshots and forgot about it.

BTW, all the above took only a few minutes. Even if you don't have an extra install to boot to, you could do the same thing from a USB stick.

Here's another way I use rollback: A couple weeks ago I was experimenting with some new software. Prior to starting, I made a fresh snapshot. Then installed some stuff, changed a bunch of stuff, and played around a bit. When I was done I didn't want to keep all that, so I moved into the BTRFS root file system. Renamed the @KDEneon subvolume and then renamed the snapshot I had taken to @KDEneon and rebooted. After I logged back in, I deleted the old @KDEneon subvolume and everything I had installed or changed was right back where I started.

[jglen490]

O.K., then perhaps I'm just lucky. I've never had any one of those situations. And having said that. I am forever doomed

I have indeed re-installed, only due to a desire to upgrade to the next release or because of a major hardware failure. And I have recovered things from backups, neither of which takes much effort or time on my part. Perhaps I don't experiment with unknowns, because I value my stable platform too much. The only experimentation I do is on my playground/laptop. Of course it's a 32 bit machine, so the play options are becoming a rather short list.

I have had difficulties with actual installs, of course, none of which would have been fixed by referring to a rollback. And that's why I stay away from Neon - not that Neon is a bad idea, but the thought of deliberately introducing instability - even if that's fixed in the next update - is completely abhorrent to me. I go from LTS to LTS on my main machine, and that's it. And I've been using Linux plus the occasional BSD for well over 20 years as my personal computing platform. Work, sadly, is another story, but as I said somewhere else the paycheck is great.

I was just curious about the rollback thing. If someone asks, I just say what I believe to be true about BTRFS or Neon or most anything else - I'm no expert and haven't written "the book" on any subject. But, if I have a question about something, it's not intended to be personal, I am just looking for information.

[oshunluvr]

Yeah, info is always good, even if you don;t want to use it. Really, the snapshot thing is marvelous.

I use BTRFS extensively on my server. Auto snapshots and backups are really a godsend when maintaining 8TB of data.

[jglen490]

And a big difference is (I'm assuming) that your 8TB of data is very likely much more dynamic than my 110GB data store, a lot of which is a pretty static collection of music and video content, with the most dynamic being email (of course) and some documents dealing with volunteer work. This being my definition of a small, single user system. A lot of that also ends up being duplicated on my Samsung tablet as well as all of the data on a rotating set of external backup drives. A very stable platform based on the standard Kubuntu LTS makes that possible and viable without needing to be a slave to my computer and not having to worry about a bunch of mess.

Thank you for the conversation!