Debugging small OS kernels (part 8)

Part of the problem with debugging in the way we describe, has to do with the fact that we’re starting and stopping the kernel all the time. For instance, if we step into the scheduler, we’ve STOPPED scheduling. This has all sorts of ramifications, which can be partially offset by using conditional breakpoints in place of hard breakpoints. It’s a little like trying to listen to the engine’s bad “ticking noise” when it’s not running. Plus, while we’re looking at a particular facet of the scheduler (while it’s “stopped” by a breakpoint) – other threads are not being serviced. Time dependent interrupts screw up, as well as other things. I’m not a kernel guru (like I said, I spent all my time in user-land code), so there may be other facets of this I’m not understanding yet. I’ve posted a message on the forum that helps to explain:

Posted by ronaldlees:

With the Visopsys build pretty solid, and QEMU working with it,
my next big thing was to figure a way to switch the current thread
in a GDB session. That's something that's real easy to do when
debugging userland programs running natively and locally.

I realized that all of the GNU debugger's process and thread
functionality was tied into specific attributes of the operating
systems (mainly of the *nix/posix types) - such as libthread and
fork, and Visopsys has neither of those. So, there are no thread
events for the debugger. GDB options like schedule-multiple,
non-stop, attach, and so forth have no meaning in this context.

Issuing an "info threads" GDB command while connected to QEMU,
while debugging Visopsys, just returns an error. The GDB server 
stub in QEMU supports debugging whatever process it started with,
but no thread info comes back when Visopsys is the virtualized OS. 
Within the initial process and "thread" - it works surprisingly well,
but it's not as useful as userland program debugging. One thing that
is an issue is that the kernel itself gets stopped and started again
and again. Sometimes, this interferes with getting useful information.
I think this setup would probably work better in a simple bare-metal
or RTOS situation, with a single task. The problem with starting and
stopping causing interference might be mitigated somewhat by using
conditional breakpoints/watchpoints instead of hard breakpoints.

I end up debugging the kernel in one session, but then have to stop
and reconnect in order to debug an application program in another
session. I tried to attach to two processes at the same time (doable
in a native/local environ) - but the GDB server stub in QEMU allows
only one connection at a time. In retrospect, that's reasonable. I
think it works in one-foot-after-the-other mode. The stub is not very
fancy. In the unfancy mode, you have one activity per CPU, and
there's only one CPU.

It does work decently to debug code in this fashion, depending on
what you want to do. When debugging an app (today I was looking at
hostname) - I can just step over any kernel function calls, make a
note for any that fail, and then restart GDB with the debugger
pointed at the kernel.

Have you ever thought about putting a serial debug output on

The last question was posted to the Visopsys’s author, relating to adding some additional debugging help that might complement the (somewhat hobbled GNU debugger approach). Using serial debug output is a little like using GDB in watchpoint-only mode. It still has its place, especially debugging kernels.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.