Are there any Wayland compositor 'toolkits'?
I'm thinking something that lets you design your own compositor by writing limited code and configuration.
Not locked into a specific paradigm and more capable than just "customize this off-the-shelf WM", but less work than rolling your own with a bunch of libraries.
(Customizable WMs *may* be suitable answers, if you can customize both their appearance and behaviour far enough that a user would not be able to identify the underlying WM from using a customized installation, as an imperfect yardstick.)