Skip to main content
julia's space

retaining qemu's terminal behaviour with non-default serial devices

Often you use a command line like this:

qemu-system-riscv64 -machine virt -kernel ./kernel -serial mon:stdio -nographic

This generates a system which uses a ns16550a-compatible serial device:

serial@10000000 {
	interrupts = <0x0a>;
	interrupt-parent = <0x03>;
	clock-frequency = "", "8@";
	reg = <0x00 0x10000000 0x00 0x100>;
	compatible = "ns16550a";
};

Let's say you have a system that wants to use VirtIO console.

tl;dr

Pass -device virtconsole,chardev=serial0 if you're happy with the behaviour of -nographic; as serial0 is the default name of the multiplexed serial output.

Add -device virtio-serial-device for an (MMIO) serial bus device, then add -device virtconsole. You may want -global virtio-mmio.force-legacy=false.

Cool. No output (on VirtIO console, at least). Of course, virtconsole needs a chardev property - so create one connect to stdio.

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -serial mon:stdio -nographic \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev stdio,the_console

Whoops.

qemu-system-riscv64: -serial mon:stdio: cannot use stdio by multiple character devices
qemu-system-riscv64: -serial mon:stdio: could not connect serial device to character backend 'mon:stdio'

Okay, let's remove -serial mon:stdio...

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -nographic \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev stdio,id=the_console

Same error? Huh.

qemu-system-riscv64: cannot use stdio by multiple character devices
qemu-system-riscv64: could not connect serial device to character backend 'mon:stdio'

What if we tell it to use the chardev as serial?

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -nographic \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev stdio,id=the_console -serial chardev:the_console

Well....

qemu-system-riscv64: cannot use stdio by multiple character devices

Uh, maybe it's because we send chardev to serial (stdio) twice? Let's send it to a PTY...

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -nographic \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev pty,id=the_console -serial chardev:the_console

Huh...

qemu-system-riscv64: -device virtconsole,chardev=the_console: Property 'virtconsole.chardev' can't take value 'the_console': chardev 'the_console' is already in use

OK, what does man qemu(1) say?

-serial dev

Redirect the virtual serial port to host character device dev. The default device is vc in graphical mode and stdio in non graphical mode.

This option can be used several times to simulate multiple serial ports.

You can use -serial none to suppress the creation of default serial devices.

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -nographic \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev pty,id=the_console -serial none

Well, it does work.... if we open a serial monitor in another window.

char device redirected to /dev/pts/9 (label the_console)
QEMU 9.2.0 monitor - type 'help' for more information
(qemu)

I discovered this bug report against QEMU where someone clearly struggled with the same issue. It received the following response:

You can start 'none' without "-serial null". Examples:

qemu-system-x86_64 -machine none
qemu-system-x86_64 -machine none -monitor stdio
qemu-system-x86_64 -machine none -nographic
qemu-system-x86_64 -machine none -monitor stdio -display none

Your command line "qemu-system-x86_64 -machine none -nographic -monitor stdio" fails because "-nographic" says "please create a serial port using stdio" but "-monitor stdio" tries to use stdio for something else. You get the same message for any machine (eg "pc"), not just "none". If what you wanted was "just don't create the graphical display" that's "-display none" -- "-nographic" is a collection of things including both 'no display' and also 'default to creating a serial device to stdio' and 'default to creating a monitor muxed with that serial'.

— Peter Maydell

And indeed, if we pass -display none, we get output:

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -display none \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev stdio,id=the_console

But there's one problem... we can't access the QEMU monitor anymore, and Ctrl+C terminates QEMU instead of being forwarded to the device.

From the question body of this stackoverflow question, we learn about a few options.

If I add:

-chardev stdio,id=s1,signal=off \
-serial none -device isa-serial,chardev=s1

then Ctrl+C starts working as desired, but Ctrl+A X does not work to quit QEMU,

Querying the QEMU manpage further, this time under -chardev, we find:

-chardev backend,id=id[,mux=on|off][,options]

Backend is one of: null, socket, udp, msmouse, hub, vc, ringbuf, file, pipe, console, serial, pty, stdio, braille, parallel, spicevmc, spiceport. The specific backend will determine the applicable options.

A character device may be used in multiplexing mode by multiple front-ends. Specify mux=on to enable this mode. A multiplexer is a “1:N” device, and here the “1” end is your specified chardev backend, and the “N” end is the various parts of QEMU that can talk to a chardev. If you create a chardev with id=myid and mux=on, QEMU will create a multiplexer with your specified ID, and you can then configure multiple front ends to use that chardev ID for their input/output. Up to four different front ends can be connected to a single multiplexed chardev. (Without multiplexing enabled, a chardev can only be used by a single front end.) For instance you could use this to allow a single stdio chardev to be used by two serial ports and the QEMU monitor:

-chardev stdio,mux=on,id=char0 \
-mon chardev=char0,mode=readline \
-serial chardev:char0 \
-serial chardev:char0

And later on:

-chardev stdio,id=id[,signal=on|off]

Connect to standard input and standard output of the QEMU process.

signal controls if signals are enabled on the terminal, that includes exiting QEMU with the key sequence Control-c. This option is enabled by default, use signal=off to disable it

So:

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -display none \
  -device virtio-serial-device -device virtconsole,chardev=the_console \
  -chardev stdio,id=the_console,mux=on,signal=off \
  -serial chardev:the_console -mon the_console

Gives us the behaviour we want.

At some point whilst writing this up, I learnt about info chardev.

(qemu) info chardev
the_console: filename=mux
the_console-base: filename=stdio
parallel0: filename=vc

And I went, "what if I just set the chardev of the virtconsole to the default serial?".

(qemu) info chardev
parallel0: filename=null
serial0: filename=mux
serial0-base: filename=stdio

And yes, the following works:

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -nographic -device virtio-serial-device \
  -device virtconsole,chardev=serial0

Yay!

Armed with the knowledge about how -nographic behaves from earlier, one can discover how it works. In qemu/system/vl.c, lines 1410-1424 (and elsewhere):

if (nographic) {
    if (default_parallel) {
        add_device_config(DEV_PARALLEL, "null");
    }
    if (default_serial && default_monitor) {
        add_device_config(DEV_SERIAL, "mon:stdio");
    } else {
        if (default_serial) {
            add_device_config(DEV_SERIAL, "stdio");
        }
        if (default_monitor) {
            monitor_parse("stdio", "readline", false);
        }
    }
} else {

So -nographic is equivalent to -display none and -serial mon:stdio unless one specifies -serial or -monitor.

This means that the following also works the same as before, if the implicitness of -nographic is confusing.

qemu-system-riscv64 -machine virt -kernel ./kernel \
  -display none -device virtio-serial-device \
  -device virtconsole,chardev=serial0 \
  -serial mon:stdio
some other miscellaneous notes

QEMU, under the Xen documentation, says this:

To provide a Xen console device, define a character device and then a device of type xen-console to connect to it. For the Xen console equivalent of the handy -serial mon:stdio option, for example:

-chardev stdio,mux=on,id=char0,signal=off -mon char0 \
-device xen-console,chardev=char0

This is essentially what we discovered all along. Thanks, QEMU documentation.

Another consequence of the default behaviour of -nographic is that I see a lot of code that passes -nographic and -serial mon:stdio, which I think is slightly silly. It likely makes more sense to pass -display none and -serial mon:stdio or just -nographic.

I'd argue that perhaps we should deprecate -nographic and advise people to use -display none, instead, because of the very confusing behaviour where passing -serial <XXX> can change how QEMU behaves in strange ways.