GitSync cannot make partial clones (git clone --filter) #553

Summary

GitSync has no way to ask for a partial clone, so tools that sync many repositories through libvcs, such as vcspull, always download every version of every file in history. Adding it also runs into two existing defects that make any filtered checkout fail through libvcs. Sized for one evening.

Problem

git (repo, docs) supports partial clones with git clone --filter=<filter-spec>: the clone keeps full commit history but leaves out objects the filter excludes, and fetches them on demand. With blob:none, file contents are downloaded only for the checkout. Measured on a local file:// clone of git.git with git 2.43.0 on Linux, a full clone took 17.5 s and 304 MB of .git; --filter=blob:none took 9.4 s and 128 MB, with every commit present.

• Git.clone, Git.fetch, Git.pull, and GitSubmoduleCmd.update accept a filter only as an undocumented _filter key popped from **kwargs (https://github.com/vcs-python/libvcs/blob/v0.46.0/src/libvcs/cmd/git.py#L370-L371). It is untyped, invisible to editors and API docs, and GitSync has no parameter for it.
• Every remote with remote.<name>.partialclonefilter set, which any filtered clone leaves behind, prints its fetch line as <name>\t<url> (fetch) [<filter-spec>] (builtin/remote.c at v2.55.0). The GitRemoteManager.ls() pattern ends right after (fetch) (https://github.com/vcs-python/libvcs/blob/v0.46.0/src/libvcs/cmd/git.py#L4418-L4428), so the fetch URL is dropped and GitSync.obtain() fails in set_remotes() with GitRemoteSetError, whether or not libvcs made the clone.
• GitSync.obtain() runs the clone without checking its exit code (https://github.com/vcs-python/libvcs/blob/v0.46.0/src/libvcs/sync/git.py#L425-L443). A clone that fails, from a bad filter, an unreachable URL, or a rejected depth, carries on into submodule setup and set_remotes(), and the error a caller sees is git remote add failing with fatal: not a git repository.

strbuf_addf(&promisor_config, "remote.%s.partialclonefilter", remote->name);
strbuf_addf(&remote_info_buf, "%s (fetch)", remote->url.v[0]);
if (!repo_config_get_string_tmp(the_repository, promisor_config.buf, &partial_clone_filter))
	strbuf_addf(&remote_info_buf, " [%s]", partial_clone_filter);

Prompt

Add partial clone support to libvcs, and fix the two defects above first, each in its own commit with a test that fails without the fix.

These must hold; everything after them is open:

• A filtered GitSync.obtain() succeeds and leaves history blobs out: git config remote.origin.partialclonefilter reports the filter, and git rev-list --objects --missing=print --all lists missing objects.
• Listing remotes keeps the fetch URL of a remote that has a partial clone filter.
• A failed clone raises from git clone, carrying git's own message.
• Existing callers that pass _filter="<spec>" keep working.
• Config-style filter data that is malformed is rejected before any clone starts.

Direction, non-binding:

• Accept an optional [<filter-spec>] suffix in GitRemoteManager.ls(), without letting the bracket match span lines.
• Pass check_returncode=True to the clone in GitSync.obtain().
• Add a libvcs.cmd.git_filter module with a frozen dataclass per filter kind (BlobNone, BlobLimit, TreeDepth, ObjectType, SparseOid, Auto, Combine) whose str() is the git spec, plus parse_filter() (spec string to model), from_mapping() (a {kind: ..., <fields>} mapping to model), coerce_filter(), and filter_specs().
• Make _filter a typed keyword on clone, fetch, pull, and GitSubmoduleCmd.update that takes a model, a spec string, or a sequence of either, and emits one --filter= per item. Add GitSync(git_filter=...), passed to the clone and to git submodule update so submodules are filtered too.
• Add a docs page next to the other git command pages, and CHANGES entries in their own commits.

Details that cost time to rediscover:

• The grammar is in list-objects-filter-options.c at v2.55.0: blob:none, blob:limit=<n>[kmg], tree:<depth>, object:type=(tag|commit|tree|blob), sparse:oid=<blob-ish>, auto, and combine:<a>+<b> with %-encoded sub-specs. sparse:path= was removed. auto needs git 2.54 or newer, only works for clone and fetch, and cannot be combined; older local git rejects it.
• git accepts combine: with a single sub-filter.
• Python's bool is an int, so a depth or size check that only tests isinstance(value, int) lets True through and renders tree:True.
• Reject mapping fields that belong to a different kind, so a typo is not silently dropped.
• Repeated --filter flags are recorded as one combine: spec. git records a lone blob:limit=1m as blob:limit=1048576 but keeps 1m inside combine:.
• A test remote needs uploadpack.allowFilter=true, and uploadpack.allowAnySHA1InWant=true for the checkout's on-demand fetch. Without them git warns and makes a full clone.
• The pytest plugin sets GIT_CONFIG (https://github.com/vcs-python/libvcs/blob/v0.46.0/src/libvcs/pytest_plugin.py#L210), which sends a plain git config key value in a test to the session-wide gitconfig. Unset it, or pass an environment without it, when a test writes git config; leaked promisor settings break unrelated remote tests later in the session.

Not doing:

• Changing an existing checkout's filter on update, which needs git fetch --refetch; that is the same class of problem as GitSync: deepen/unshallow on update when depth changes (follow-up to #531) #532.
• Checking whether a server supports a filter.

Alternatives

Weighed and set aside:

• A Literal[...] | str alias alone: editors complete the common values, but type checkers treat it as str and nothing validates config-style data.
• kind-tagged TypedDicts: pydantic rejects typing.TypedDict before Python 3.12, so a consumer validating them with pydantic breaks on 3.10 and 3.11 unless libvcs takes on typing_extensions.
• A str subclass with builder classmethods: good call sites, but no mapping form for config files.

References

• git clone --filter and the filter-spec forms
• Partial clone design notes at v2.55.0
• GitSync: honor arbitrary clone depth on sync, not just --depth 1 #531 and GitSync: deepen/unshallow on update when depth changes (follow-up to #531) #532, arbitrary clone depth and changing depth on update