Follow

: I have a Rust question. My usual approach to developing a collection of modular (but related) libraries in JS is:

1. Start with one monolithic package, in which everything is developed initially.
2. But opportunistically minimize assumptions between parts of the package, so that internally it is already mostly modular.
3. Spend some time figuring out reasonable APIs for the remaining, more-difficult-to-modularize cases.
4. Eventually split out all the internal parts into separate libraries/packages and publish them independently, so that users can select only the parts that they need, and easily combine it with third-party libraries.

What would a good workflow for this look like in Rust, especially in terms of internal crate layout (in step 1) so that it's easy to split out into separate crates later, but also in terms of tools to use?

I do not use monorepos, each crate will have its own repository in the end. Also, contributors must be able to use standard Cargo commands to manage the different crates, so no 'wrapper scripts' or third-party tools (unless they can be made to run from within Cargo).

:boost_requested:

· · Web · 3 · 2 · 1

@joepie91 I don't know what tools to use (because so far, I've only done such stuff manually).

Not using a monorepo seems to make this more difficult (but one could start with a monorepo and get rid of it / replace it with individual repos when 4. is done). For that transformation, github.com/newren/git-filter-r is really useful.

It's what I've used for similar splits in the past, e.g. extracting codeberg.org/topola/contracts- from codeberg.org/topola/topola.
For another submodule, (2.) codeberg.org/topola/topola/com

@joepie91 The initial crate layout basically has what would later be separate packages as submodules, and there are some attempts to minimize circular dependencies between them. (unfortunately, in the Topola case, the usage of `enum-dispatch` turned out to make that really difficult, so one has to be somewhat attentive to how one chooses utility procedural macros)

@joepie91

I did that once with a crate that grew as a subsystem of a different library and cargo already did everything that was needed, the rest is writing code.

This is a mix of what I did and lessons learned:

For internal crate layout my recommendation would be to already put the part you eventually want to split off into its own module and try to keep the dependencies unidirectional. (No magic here)

Start documenting as early as possible (cargo doc is a huge help, even for yourself), especially the future library surface, but also the private internals of a crate. This doesn’t have to be very detailed, but it should include assumptions, intended use and non obvious (most) error cases.

If you want to go soft with the final splitting off you may want to use a cargo workspace where you have the crates still in the same repo, but already as different crates. This can also help with untangling dependencies. (Only worth the effort if there is a lot to untangle).

Another trick I use is to temporarily point to the local development version of a crate using a path reference in Cargo.toml which is a huge help when trying to evolve a library together with its main usecase. When the library crate gets the release I point it back at the crate registry.

I hope this is at least part of the answer you tried to get :drgn_flat_happy:

@joepie91 And remember the fact that traits can be used to implement extra functionality on types from other crates (if the trait is in your crate) to minimize circular dependencies if that is what you need.

@joepie91 Such granularity in packaging is something I do not recognize from Rust. Packaging as a monolith is more common.

If you have modules for interoperability with other, optional, packages you could solve this with Cargo features?

Sign in to participate in the conversation
Pixietown

Small server part of the pixie.town infrastructure. Registration is closed.