Linus on Debuggers and Kernel Development
Sep 07, 2000, 14:26 (31 Talkback[s])
Re-Imagining Linux Platforms to Meet the Needs of Cloud Service Providers
From the Linux Kernel Mailing List:
Date: Wed, 6 Sep 2000 12:52:29 -0700 (PDT)
From: Linus Torvalds
Subject: Re: Availability of kdb
On Wed, 6 Sep 2000, Tigran Aivazian wrote:
> very nice monologue, thanks. It would be great to know Linus' opinion. I
> mean, I knew Linus' opinion of some years' ago but perhaps it changed? He
> is a living being and not some set of rules written in stone so perhaps
> current stability/highquality of kdb suggests to Linus that it may be
> (just maybe) acceptable into official tree?
I don't like debuggers. Never have, probably never will. I use gdb all the
time, but I tend to use it not as a debugger, but as a disassembler on
steroids that you can program.
None of the arguments for a kernel debugger has touched me in the least.
And trust me, over the years I've heard quite a lot of them. In the end,
they tend to boil down to basically:
- it would be so much easier to do development, and we'd be able to add
new things faster.
And quite frankly, I don't care. I don't think kernel development should
be "easy". I do not condone single-stepping through code to find the bug.
I do not think that extra visibility into the system is necessarily a good
Apparently, if you follow the arguments, not having a kernel debugger
leads to various maladies:
- you crash when something goes wrong, and you fsck and it takes forever
and you get frustrated.
- people have given up on Linux kernel programming because it's too hard
and too time-consuming
- it takes longer to create new features.
And nobody has explained to me why these are _bad_ things.
To me, it's not a bug, it's a feature. Not only is it documented, but it's
_good_, so it obviously cannot be a bug.
"Takes longer to create new features" - this one in particular is not a
very strong argument for having a debugger. It's not as if lack of
features or new code would be a problem for Linux, or, in fact, for the
software industry as a whole. Quite the reverse. My biggest job is to say
"no" to new features, not trying to find them.
Oh. And sure, when things crash and you fsck and you didn't even get a
clue about what went wrong, you get frustrated. Tough. There are two kinds
of reactions to that: you start being careful, or you start whining about
a kernel debugger.
Quite frankly, I'd rather weed out the people who don't start being
careful early rather than late. That sounds callous, and by God, it _is_
callous. But it's not the kind of "if you can't stand the heat, get out
the the kitchen" kind of remark that some people take it for. No, it's
something much more deeper: I'd rather not work with people who aren't
careful. It's darwinism in software development.
It's a cold, callous argument that says that there are two kinds of
people, and I'd rather not work with the second kind. Live with it.
I'm a bastard. I have absolutely no clue why people can ever think
otherwise. Yet they do. People think I'm a nice guy, and the fact is that
I'm a scheming, conniving bastard who doesn't care for any hurt feelings
or lost hours of work if it just results in what I consider to be a better
And I'm not just saying that. I'm really not a very nice person. I can say
"I don't care" with a straight face, and really mean it.
I happen to believe that not having a kernel debugger forces people to
think about their problem on a different level than with a debugger. I
think that without a debugger, you don't get into that mindset where you
know how it behaves, and then you fix it from there. Without a debugger,
you tend to think about problems another way. You want to understand
things on a different _level_.
It's partly "source vs binary", but it's more than that. It's not that you
have to look at the sources (of course you have to - and any good debugger
will make that _easy_). It's that you have to look at the level _above_
sources. At the meaning of things. Without a debugger, you basically have
to go the next step: understand what the program does. Not just that
And quite frankly, for most of the real problems (as opposed to the stupid
bugs - of which there are many, as the latest crap with "truncate()" has
shown us) a debugger doesn't much help. And the real problems are what I
worry about. The rest is just details. It will get fixed eventually.
I do realize that others disagree. And I'm not your Mom. You can use a
kernel debugger if you want to, and I won't give you the cold shoulder
because you have "sullied" yourself. But I'm not going to help you use
one, and I wuld frankly prefer people not to use kernel debuggers that
much. So I don't make it part of the standard distribution, and if the
existing debuggers aren't very well known I won't shed a tear over it.
Because I'm a bastard, and proud of it!