Let guest applications can communicate with host applications.

Use POSIX Sockets API: A new socket family AF_VSOCK is hereby introduced.

This can be used to implement hypervisor services and guest agents.

  • 可以 Guest App 和 Host Service 交互，也可以用来让 Host 的 App 之间进行交互；
  • 在 kata-container 中经常用到；
  • 每个 VM 都有固定的 cid，port 是 user app 自己指定

Virtio-vsock: 沟通VM内外的桥梁virtio-vsock是一种专门用于Guest VM和Host OS交互 - 掘金

Host services communicate with each other using VSock

可以看到连接的原语和普通的 socket 无异。

Server code:

import socket
# 使用 host 的 CID，类似于 localhost
CID = socket.VMADDR_CID_HOST
PORT = 9999
s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
s.bind((CID, PORT))
s.listen()
# socket cid 和 port 都是随机的
(conn, (remote_cid, remote_port)) = s.accept()
print(f"Connection opened by cid={remote_cid} port={remote_port}")
while True:
    buf = conn.recv(64)
    if not buf:
        breakprint(f"Received bytes: {buf}")

Client code:

import socket
# 使用 host 的 CID，类似于 localhost
CID = socket.VMADDR_CID_HOST
PORT = 9999
s = socket.socket(socket.AF_VSOCK, socket.SOCK_STREAM)
# 连接 server 端的 9999 端口，自己的端口是随机的。
s.connect((CID, PORT))
s.sendall(b"Hello, world!")
s.close()

Limitation of virtio-serial

N:1 connections are clunky over 1:1 serial port

  • Applications have to multiplex over 1 stream
  • Libvirt has to arbitrate access between qemu-guest-agent clients Relatively low number of ports available (~512)
  • Limit is hardcoded by host Stream semantics (no message boundaries)
  • Ugly for datagram protocols

Applications must use character devices instead of familiar sockets API…

stefanha-kvm-forum-2015.pdf

CID

You can think it like the IP address.

The host and each VM have a 32 bit CID (Context IDentifier) and may connect or bind to a 32 bit port number.

Virtio-vsock / Vhost-vsock-pci

virtio-vsock is a vhost-based virtio device.

virtio-vsock - tycoon3 - 博客园

Both virtio-vsock and vhost-vsock-pci are virtual socket devices used to facilitate communication between a virtual machine and the host system in a virtualized environment. However, there are some differences between them.

Virtio-vsock is a virtual socket device that uses the virtio framework. It is designed to be efficient and secure, with minimal CPU overhead and strong isolation between the guest and host systems.

On the other hand, vhost-vsock-pci is a virtual socket device that uses the vhost framework, which is a Linux kernel module used for virtio-based virtualization. Vhost-vsock-pci is a more recent addition to the Linux kernel, it is specifically designed to work with the PCI bus, which may provide some performance advantages in certain scenarios.

In summary, Virtio-vsock uses the virtio framework, while vhost-vsock-pci uses the vhost framework and is specifically designed to work with the PCI bus.

VSock Example

sudo modprobe vhost_vsock

To create a VM with a vsock device with CID 123:

qemu-system-x86_64 -device vhost-vsock-pci,guest-cid=123

Example of guest as client, host as server: vsock notes (Note, guest can also be the server and host as the client, see the comment here).