diff --git a/Documentation/config.adoc b/Documentation/config.adoc index 62eebe7c54501c..dcea3c0c15e2a9 100644 --- a/Documentation/config.adoc +++ b/Documentation/config.adoc @@ -523,6 +523,8 @@ include::config/sequencer.adoc[] include::config/showbranch.adoc[] +include::config/sideband.adoc[] + include::config/sparse.adoc[] include::config/splitindex.adoc[] diff --git a/Documentation/config/difftool.adoc b/Documentation/config/difftool.adoc index 4f7d40ce242b78..1b8d48381357aa 100644 --- a/Documentation/config/difftool.adoc +++ b/Documentation/config/difftool.adoc @@ -1,43 +1,43 @@ -diff.tool:: +`diff.tool`:: Controls which diff tool is used by linkgit:git-difftool[1]. This variable overrides the value configured in `merge.tool`. The list below shows the valid built-in values. Any other value is treated as a custom diff tool and requires - that a corresponding difftool..cmd variable is defined. + that a corresponding `difftool..cmd` variable is defined. -diff.guitool:: +`diff.guitool`:: Controls which diff tool is used by linkgit:git-difftool[1] when - the -g/--gui flag is specified. This variable overrides the value + the `-g`/`--gui` flag is specified. This variable overrides the value configured in `merge.guitool`. The list below shows the valid built-in values. Any other value is treated as a custom diff tool - and requires that a corresponding difftool..cmd variable + and requires that a corresponding `difftool..cmd` variable is defined. include::{build_dir}/mergetools-diff.adoc[] -difftool..cmd:: +`difftool..cmd`:: Specify the command to invoke the specified diff tool. The specified command is evaluated in shell with the following - variables available: 'LOCAL' is set to the name of the temporary - file containing the contents of the diff pre-image and 'REMOTE' + variables available: `LOCAL` is set to the name of the temporary + file containing the contents of the diff pre-image and `REMOTE` is set to the name of the temporary file containing the contents of the diff post-image. + See the `--tool=` option in linkgit:git-difftool[1] for more details. -difftool..path:: +`difftool..path`:: Override the path for the given tool. This is useful in case your tool is not in the PATH. -difftool.trustExitCode:: +`difftool.trustExitCode`:: Exit difftool if the invoked diff tool returns a non-zero exit status. + See the `--trust-exit-code` option in linkgit:git-difftool[1] for more details. -difftool.prompt:: +`difftool.prompt`:: Prompt before each invocation of the diff tool. -difftool.guiDefault:: +`difftool.guiDefault`:: Set `true` to use the `diff.guitool` by default (equivalent to specifying the `--gui` argument), or `auto` to select `diff.guitool` or `diff.tool` depending on the presence of a `DISPLAY` environment variable value. The diff --git a/Documentation/config/hook.adoc b/Documentation/config/hook.adoc index 9e78f264396ca5..a9dc0063c12102 100644 --- a/Documentation/config/hook.adoc +++ b/Documentation/config/hook.adoc @@ -15,6 +15,12 @@ hook..event:: events, specify the key more than once. An empty value resets the list of events, clearing any previously defined events for `hook.`. See linkgit:git-hook[1]. ++ +The `` must not be the same as a known hook event name +(e.g. do not use `hook.pre-commit.event`). Using a known event name as +a friendly-name is a fatal error because it creates an ambiguity with +`hook..enabled` and `hook..jobs`. For unknown event names, +a warning is issued when `` matches the event value. hook..enabled:: Whether the hook `hook.` is enabled. Defaults to `true`. @@ -22,3 +28,73 @@ hook..enabled:: configuration. This is particularly useful when a hook is defined in a system or global config file and needs to be disabled for a specific repository. See linkgit:git-hook[1]. + +hook..parallel:: + Whether the hook `hook.` may run in parallel with other hooks + for the same event. Defaults to `false`. Set to `true` only when the + hook script is safe to run concurrently with other hooks for the same + event. If any hook for an event does not have this set to `true`, + all hooks for that event run sequentially regardless of `hook.jobs`. + Only configured (named) hooks need to declare this. Traditional hooks + found in the hooks directory do not need to, and run in parallel when + the effective job count is greater than 1. See linkgit:git-hook[1]. + +hook..enabled:: + Switch to enable or disable all hooks for the `` hook event. + When set to `false`, no hooks fire for that event, regardless of any + per-hook `hook..enabled` settings. Defaults to `true`. + See linkgit:git-hook[1]. ++ +Note on naming: `` must be the event name (e.g. `pre-commit`), +not a hook friendly-name. Since using a known event name as a +friendly-name is disallowed (see `hook..event` above), +there is no ambiguity between event-level and per-hook `.enabled` +settings for known events. For unknown events, if a friendly-name +matches the event name despite the warning, `.enabled` is treated +as per-hook only. + +hook..jobs:: + Specifies how many hooks can be run simultaneously for the `` + hook event (e.g. `hook.post-receive.jobs = 4`). Overrides `hook.jobs` + for this specific event. The same parallelism restrictions apply: this + setting has no effect unless all configured hooks for the event have + `hook..parallel` set to `true`. Set to `-1` to use the + number of available CPU cores. Must be a positive integer or `-1`; + zero is rejected with a warning. See linkgit:git-hook[1]. ++ +Note on naming: although this key resembles `hook..*` +(a per-hook setting), `` must be the event name, not a hook +friendly name. The key component is stored literally and looked up by +event name at runtime with no translation between the two namespaces. +A key like `hook.my-hook.jobs` is stored under `"my-hook"` but the +lookup at runtime uses the event name (e.g. `"post-receive"`), so +`hook.my-hook.jobs` is silently ignored even when `my-hook` is +registered for that event. Use `hook.post-receive.jobs` or any other +valid event name when setting `hook..jobs`. + +hook.jobs:: + Specifies how many hooks can be run simultaneously during parallelized + hook execution. If unspecified, defaults to 1 (serial execution). + Set to `-1` to use the number of available CPU cores. + Can be overridden on a per-event basis with `hook..jobs`. + Some hooks always run sequentially regardless of this setting because + they operate on shared data and cannot safely be parallelized: ++ +-- +`applypatch-msg`;; +`prepare-commit-msg`;; +`commit-msg`;; + Receive a commit message file and may rewrite it in place. +`pre-commit`;; +`post-checkout`;; +`push-to-checkout`;; +`post-commit`;; + Access the working tree, index, or repository state. +-- ++ +This setting has no effect unless all configured hooks for the event have +`hook..parallel` set to `true`. ++ +For `pre-push` hooks, which normally keep stdout and stderr separate, +setting this to a value greater than 1 (or passing `-j`) will merge stdout +into stderr to allow correct de-interleaving of parallel output. diff --git a/Documentation/config/mergetool.adoc b/Documentation/config/mergetool.adoc index 7064f5a462cb56..7afdcad92b3934 100644 --- a/Documentation/config/mergetool.adoc +++ b/Documentation/config/mergetool.adoc @@ -52,13 +52,13 @@ if `merge.tool` is configured as __), Git will consult `mergetool..layout` to determine the tool's layout. If the variant-specific configuration is not available, `vimdiff` ' s is used as - fallback. If that too is not available, a default layout with 4 windows - will be used. To configure the layout, see the 'BACKEND SPECIFIC HINTS' + fallback. If that too is not available, a default layout with 4 windows + will be used. ifdef::git-mergetool[] - section. +To configure the layout, see the 'BACKEND SPECIFIC HINTS' section. endif::[] ifndef::git-mergetool[] - section in linkgit:git-mergetool[1]. +To configure the layout, see the 'BACKEND SPECIFIC HINTS' section in linkgit:git-mergetool[1]. endif::[] `mergetool.hideResolved`:: diff --git a/Documentation/config/sideband.adoc b/Documentation/config/sideband.adoc new file mode 100644 index 00000000000000..96fade7f5fee39 --- /dev/null +++ b/Documentation/config/sideband.adoc @@ -0,0 +1,27 @@ +sideband.allowControlCharacters:: + By default, control characters that are delivered via the sideband + are masked, except ANSI color sequences. This prevents potentially + unwanted ANSI escape sequences from being sent to the terminal. Use + this config setting to override this behavior (the value can be + a comma-separated list of the following keywords): ++ +-- + `color`:: + Allow ANSI color sequences, line feeds and horizontal tabs, + but mask all other control characters. This is the default. + `cursor:`: + Allow control sequences that move the cursor. This is + disabled by default. + `erase`:: + Allow control sequences that erase charactrs. This is + disabled by default. + `false`:: + Mask all control characters other than line feeds and + horizontal tabs. + `true`:: + Allow all control characters to be sent to the terminal. +-- + +sideband..*:: + Apply the `sideband.*` option selectively to specific URLs. The + same URL matching logic applies as for `http..*` settings. diff --git a/Documentation/git-describe.adoc b/Documentation/git-describe.adoc index 08ff715709ccd1..b2cb1e47e46c67 100644 --- a/Documentation/git-describe.adoc +++ b/Documentation/git-describe.adoc @@ -7,10 +7,10 @@ git-describe - Give an object a human readable name based on an available ref SYNOPSIS -------- -[verse] -'git describe' [--all] [--tags] [--contains] [--abbrev=] [...] -'git describe' [--all] [--tags] [--contains] [--abbrev=] --dirty[=] -'git describe' +[synopsis] +git describe [--all] [--tags] [--contains] [--abbrev=] [...] +git describe [--all] [--tags] [--contains] [--abbrev=] --dirty[=] +git describe DESCRIPTION ----------- @@ -22,70 +22,70 @@ abbreviated object name of the most recent commit. The result is a "human-readable" object name which can also be used to identify the commit to other git commands. -By default (without --all or --tags) `git describe` only shows +By default (without `--all` or `--tags`) `git describe` only shows annotated tags. For more information about creating annotated tags -see the -a and -s options to linkgit:git-tag[1]. +see the `-a` and `-s` options to linkgit:git-tag[1]. If the given object refers to a blob, it will be described as `:`, such that the blob can be found -at `` in the ``, which itself describes the +at __ in the __, which itself describes the first commit in which this blob occurs in a reverse revision walk -from HEAD. +from `HEAD`. OPTIONS ------- -...:: - Commit-ish object names to describe. Defaults to HEAD if omitted. +`...`:: + Commit-ish object names to describe. Defaults to `HEAD` if omitted. ---dirty[=]:: ---broken[=]:: +`--dirty[=]`:: +`--broken[=]`:: Describe the state of the working tree. When the working - tree matches HEAD, the output is the same as "git describe - HEAD". If the working tree has local modification "-dirty" + tree matches `HEAD`, the output is the same as `git describe HEAD`. + If the working tree has local modification, `-dirty` is appended to it. If a repository is corrupt and Git cannot determine if there is local modification, Git will - error out, unless `--broken' is given, which appends - the suffix "-broken" instead. + error out, unless `--broken` is given, which appends + the suffix `-broken` instead. ---all:: +`--all`:: Instead of using only the annotated tags, use any ref found in `refs/` namespace. This option enables matching any known branch, remote-tracking branch, or lightweight tag. ---tags:: +`--tags`:: Instead of using only the annotated tags, use any tag found in `refs/tags` namespace. This option enables matching a lightweight (non-annotated) tag. ---contains:: +`--contains`:: Instead of finding the tag that predates the commit, find the tag that comes after the commit, and thus contains it. - Automatically implies --tags. + Automatically implies `--tags`. ---abbrev=:: +`--abbrev=`:: Instead of using the default number of hexadecimal digits (which will vary according to the number of objects in the repository with - a default of 7) of the abbreviated object name, use digits, or - as many digits as needed to form a unique object name. An of 0 + a default of 7) of the abbreviated object name, use __ digits, or + as many digits as needed to form a unique object name. An __ of 0 will suppress long format, only showing the closest tag. ---candidates=:: +`--candidates=`:: Instead of considering only the 10 most recent tags as candidates to describe the input commit-ish consider - up to candidates. Increasing above 10 will take + up to __ candidates. Increasing __ above 10 will take slightly longer but may produce a more accurate result. - An of 0 will cause only exact matches to be output. + An __ of 0 will cause only exact matches to be output. ---exact-match:: +`--exact-match`:: Only output exact matches (a tag directly references the - supplied commit). This is a synonym for --candidates=0. + supplied commit). This is a synonym for `--candidates=0`. ---debug:: +`--debug`:: Verbosely display information about the searching strategy being employed to standard error. The tag name will still be printed to standard out. ---long:: +`--long`:: Always output the long format (the tag, the number of commits and the abbreviated commit name) even when it matches a tag. This is useful when you want to see parts of the commit object name @@ -94,8 +94,8 @@ OPTIONS describe such a commit as v1.2-0-gdeadbee (0th commit since tag v1.2 that points at object deadbee....). ---match :: - Only consider tags matching the given `glob(7)` pattern, +`--match `:: + Only consider tags matching the given `glob`(7) pattern, excluding the "refs/tags/" prefix. If used with `--all`, it also considers local branches and remote-tracking references matching the pattern, excluding respectively "refs/heads/" and "refs/remotes/" @@ -104,22 +104,22 @@ OPTIONS matching any of the patterns will be considered. Use `--no-match` to clear and reset the list of patterns. ---exclude :: - Do not consider tags matching the given `glob(7)` pattern, excluding +`--exclude `:: + Do not consider tags matching the given `glob`(7) pattern, excluding the "refs/tags/" prefix. If used with `--all`, it also does not consider local branches and remote-tracking references matching the pattern, - excluding respectively "refs/heads/" and "refs/remotes/" prefix; + excluding respectively "`refs/heads/`" and "`refs/remotes/`" prefix; references of other types are never considered. If given multiple times, a list of patterns will be accumulated and tags matching any of the - patterns will be excluded. When combined with --match a tag will be - considered when it matches at least one --match pattern and does not - match any of the --exclude patterns. Use `--no-exclude` to clear and + patterns will be excluded. When combined with `--match` a tag will be + considered when it matches at least one `--match` pattern and does not + match any of the `--exclude` patterns. Use `--no-exclude` to clear and reset the list of patterns. ---always:: +`--always`:: Show uniquely abbreviated commit object as fallback. ---first-parent:: +`--first-parent`:: Follow only the first parent commit upon seeing a merge commit. This is useful when you wish to not match tags on branches merged in the history of the target commit. @@ -139,8 +139,8 @@ an abbreviated object name for the commit itself ("2414721") at the end. The number of additional commits is the number -of commits which would be displayed by "git log v1.0.4..parent". -The hash suffix is "-g" + an unambiguous abbreviation for the tip commit +of commits which would be displayed by `git log v1.0.4..parent`. +The hash suffix is "`-g`" + an unambiguous abbreviation for the tip commit of parent (which was `2414721b194453f058079d897d13c4e377f92dc6`). The length of the abbreviation scales as the repository grows, using the approximate number of objects in the repository and a bit of math @@ -149,12 +149,12 @@ The "g" prefix stands for "git" and is used to allow describing the version of a software depending on the SCM the software is managed with. This is useful in an environment where people may use different SCMs. -Doing a 'git describe' on a tag-name will just show the tag name: +Doing a `git describe` on a tag-name will just show the tag name: [torvalds@g5 git]$ git describe v1.0.4 v1.0.4 -With --all, the command can use branch heads as references, so +With `--all`, the command can use branch heads as references, so the output shows the reference path as well: [torvalds@g5 git]$ git describe --all --abbrev=4 v1.0.5^2 @@ -163,7 +163,7 @@ the output shows the reference path as well: [torvalds@g5 git]$ git describe --all --abbrev=4 HEAD^ heads/lt/describe-7-g975b -With --abbrev set to 0, the command can be used to find the +With `--abbrev` set to 0, the command can be used to find the closest tagname without any suffix: [torvalds@g5 git]$ git describe --abbrev=0 v1.0.5^2 @@ -179,13 +179,13 @@ be sufficient to disambiguate these commits. SEARCH STRATEGY --------------- -For each commit-ish supplied, 'git describe' will first look for +For each commit-ish supplied, `git describe` will first look for a tag which tags exactly that commit. Annotated tags will always be preferred over lightweight tags, and tags with newer dates will always be preferred over tags with older dates. If an exact match is found, its name will be output and searching will stop. -If an exact match was not found, 'git describe' will walk back +If an exact match was not found, `git describe` will walk back through the commit history to locate an ancestor commit which has been tagged. The ancestor's tag will be output along with an abbreviation of the input commit-ish's SHA-1. If `--first-parent` was @@ -203,7 +203,7 @@ BUGS Tree objects as well as tag objects not pointing at commits, cannot be described. When describing blobs, the lightweight tags pointing at blobs are ignored, -but the blob is still described as : despite the lightweight +but the blob is still described as `:` despite the lightweight tag being favorable. GIT diff --git a/Documentation/git-difftool.adoc b/Documentation/git-difftool.adoc index 064bc683471f21..dd7cacf95e35df 100644 --- a/Documentation/git-difftool.adoc +++ b/Documentation/git-difftool.adoc @@ -7,64 +7,64 @@ git-difftool - Show changes using common diff tools SYNOPSIS -------- -[verse] -'git difftool' [] [ []] [--] [...] +[synopsis] +git difftool [] [ []] [--] [...] DESCRIPTION ----------- -'git difftool' is a Git command that allows you to compare and edit files -between revisions using common diff tools. 'git difftool' is a frontend -to 'git diff' and accepts the same options and arguments. See +`git difftool` is a Git command that allows you to compare and edit files +between revisions using common diff tools. `git difftool` is a frontend +to `git diff` and accepts the same options and arguments. See linkgit:git-diff[1]. OPTIONS ------- --d:: ---dir-diff:: +`-d`:: +`--dir-diff`:: Copy the modified files to a temporary location and perform a directory diff on them. This mode never prompts before launching the diff tool. --y:: ---no-prompt:: +`-y`:: +`--no-prompt`:: Do not prompt before launching a diff tool. ---prompt:: +`--prompt`:: Prompt before each invocation of the diff tool. This is the default behaviour; the option is provided to override any configuration settings. ---rotate-to=:: - Start showing the diff for the given path, +`--rotate-to=`:: + Start showing the diff for __, the paths before it will move to the end and output. ---skip-to=:: - Start showing the diff for the given path, skipping all +`--skip-to=`:: + Start showing the diff for __, skipping all the paths before it. --t :: ---tool=:: - Use the diff tool specified by . Valid values include +`-t `:: +`--tool=`:: + Use the diff tool specified by __. Valid values include emerge, kompare, meld, and vimdiff. Run `git difftool --tool-help` - for the list of valid settings. + for the list of valid __ settings. + -If a diff tool is not specified, 'git difftool' +If a diff tool is not specified, `git difftool` will use the configuration variable `diff.tool`. If the -configuration variable `diff.tool` is not set, 'git difftool' +configuration variable `diff.tool` is not set, `git difftool` will pick a suitable default. + You can explicitly provide a full path to the tool by setting the configuration variable `difftool..path`. For example, you can configure the absolute path to kdiff3 by setting -`difftool.kdiff3.path`. Otherwise, 'git difftool' assumes the +`difftool.kdiff3.path`. Otherwise, `git difftool` assumes the tool is available in PATH. + Instead of running one of the known diff tools, -'git difftool' can be customized to run an alternative program +`git difftool` can be customized to run an alternative program by specifying the command line to invoke in a configuration variable `difftool..cmd`. + -When 'git difftool' is invoked with this tool (either through the +When `git difftool` is invoked with this tool (either through the `-t` or `--tool` option or the `diff.tool` configuration variable) the configured command line will be invoked with the following variables available: `$LOCAL` is set to the name of the temporary @@ -74,30 +74,30 @@ of the diff post-image. `$MERGED` is the name of the file which is being compared. `$BASE` is provided for compatibility with custom merge tool commands and has the same value as `$MERGED`. ---tool-help:: +`--tool-help`:: Print a list of diff tools that may be used with `--tool`. ---symlinks:: ---no-symlinks:: - 'git difftool''s default behavior is to create symlinks to the +`--symlinks`:: +`--no-symlinks`:: + `git difftool`'s default behavior is to create symlinks to the working tree when run in `--dir-diff` mode and the right-hand side of the comparison yields the same content as the file in the working tree. + -Specifying `--no-symlinks` instructs 'git difftool' to create copies +Specifying `--no-symlinks` instructs `git difftool` to create copies instead. `--no-symlinks` is the default on Windows. --x :: ---extcmd=:: +`-x `:: +`--extcmd=`:: Specify a custom command for viewing diffs. - 'git-difftool' ignores the configured defaults and runs + `git-difftool` ignores the configured defaults and runs ` $LOCAL $REMOTE` when this option is specified. Additionally, `$BASE` is set in the environment. --g:: ---gui:: ---no-gui:: - When 'git-difftool' is invoked with the `-g` or `--gui` option +`-g`:: +`--gui`:: +`--no-gui`:: + When `git-difftool` is invoked with the `-g` or `--gui` option the default diff tool will be read from the configured `diff.guitool` variable instead of `diff.tool`. This may be selected automatically using the configuration variable @@ -106,20 +106,20 @@ instead. `--no-symlinks` is the default on Windows. fallback in the order of `merge.guitool`, `diff.tool`, `merge.tool` until a tool is found. ---trust-exit-code:: ---no-trust-exit-code:: +`--trust-exit-code`:: +`--no-trust-exit-code`:: Errors reported by the diff tool are ignored by default. - Use `--trust-exit-code` to make 'git-difftool' exit when an + Use `--trust-exit-code` to make `git-difftool` exit when an invoked diff tool returns a non-zero exit code. + -'git-difftool' will forward the exit code of the invoked tool when +`git-difftool` will forward the exit code of the invoked tool when `--trust-exit-code` is used. See linkgit:git-diff[1] for the full list of supported options. CONFIGURATION ------------- -'git difftool' falls back to 'git mergetool' config variables when the +`git difftool` falls back to `git mergetool` config variables when the difftool equivalents have not been defined. include::includes/cmd-config-section-rest.adoc[] diff --git a/Documentation/git-hook.adoc b/Documentation/git-hook.adoc index 318c637bd8eba5..46ea52db55f268 100644 --- a/Documentation/git-hook.adoc +++ b/Documentation/git-hook.adoc @@ -8,7 +8,8 @@ git-hook - Run git hooks SYNOPSIS -------- [verse] -'git hook' run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [-- ] +'git hook' run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [(-j|--jobs) ] + [-- ] 'git hook' list [--allow-unknown-hook-name] [-z] [--show-scope] DESCRIPTION @@ -147,6 +148,23 @@ OPTIONS mirroring the output style of `git config --show-scope`. Traditional hooks from the hookdir are unaffected. +-j:: +--jobs:: + Only valid for `run`. ++ +Specify how many hooks to run simultaneously. If this flag is not specified, +the value of the `hook.jobs` config is used, see linkgit:git-config[1]. If +neither is specified, defaults to 1 (serial execution). ++ +When greater than 1, it overrides the per-hook `hook..parallel` +setting, allowing all hooks for the event to run concurrently, even if they +are not individually marked as parallel. ++ +Some hooks always run sequentially regardless of this flag or the +`hook.jobs` config, because git knows they cannot safely run in parallel: +`applypatch-msg`, `pre-commit`, `prepare-commit-msg`, `commit-msg`, +`post-commit`, `post-checkout`, and `push-to-checkout`. + WRAPPERS -------- @@ -169,7 +187,8 @@ running: git hook run --allow-unknown-hook-name mywrapper-start-tests \ # providing something to stdin --stdin some-tempfile-123 \ - # execute hooks in serial + # execute multiple hooks in parallel + --jobs 3 \ # plus some arguments of your own... -- \ --testname bar \ diff --git a/Documentation/git-range-diff.adoc b/Documentation/git-range-diff.adoc index b5e85d37f1bee7..880557084533fb 100644 --- a/Documentation/git-range-diff.adoc +++ b/Documentation/git-range-diff.adoc @@ -7,8 +7,8 @@ git-range-diff - Compare two commit ranges (e.g. two versions of a branch) SYNOPSIS -------- -[verse] -'git range-diff' [--color=[]] [--no-color] [] +[synopsis] +git range-diff [--color=[]] [--no-color] [] [--no-dual-color] [--creation-factor=] [--left-only | --right-only] [--diff-merges=] [--remerge-diff] @@ -21,14 +21,14 @@ DESCRIPTION This command shows the differences between two versions of a patch series, or more generally, two commit ranges (ignoring merge commits). -In the presence of `` arguments, these commit ranges are limited +In the presence of __ arguments, these commit ranges are limited accordingly. To that end, it first finds pairs of commits from both commit ranges that correspond with each other. Two commits are said to correspond when the diff between their patches (i.e. the author information, the commit message and the commit diff) is reasonably small compared to the -patches' size. See ``Algorithm`` below for details. +patches' size. See 'Algorithm' below for details. Finally, the list of matching commits is shown in the order of the second commit range, with unmatched commits being inserted just after @@ -37,7 +37,7 @@ all of their ancestors have been shown. There are three ways to specify the commit ranges: - ` `: Either commit range can be of the form - `..`, `^!` or `^-`. See `SPECIFYING RANGES` + `..`, `^!` or `^-`. See 'SPECIFYING RANGES' in linkgit:gitrevisions[7] for more details. - `...`. This is equivalent to @@ -48,7 +48,7 @@ There are three ways to specify the commit ranges: OPTIONS ------- ---no-dual-color:: +`--no-dual-color`:: When the commit diffs differ, `git range-diff` recreates the original diffs' coloring, and adds outer -/+ diff markers with the *background* being red/green to make it easier to see e.g. @@ -56,33 +56,33 @@ OPTIONS + Additionally, the commit diff lines that are only present in the first commit range are shown "dimmed" (this can be overridden using the `color.diff.` -config setting where `` is one of `contextDimmed`, `oldDimmed` and +config setting where __ is one of `contextDimmed`, `oldDimmed` and `newDimmed`), and the commit diff lines that are only present in the second commit range are shown in bold (which can be overridden using the config -settings `color.diff.` with `` being one of `contextBold`, +settings `color.diff.` with __ being one of `contextBold`, `oldBold` or `newBold`). + This is known to `range-diff` as "dual coloring". Use `--no-dual-color` to revert to color all lines according to the outer diff markers (and completely ignore the inner diff when it comes to color). ---creation-factor=:: - Set the creation/deletion cost fudge factor to ``. +`--creation-factor=`:: + Set the creation/deletion cost fudge factor to __. Defaults to 60. Try a larger value if `git range-diff` erroneously considers a large change a total rewrite (deletion of one commit and addition of another), and a smaller one in the reverse case. - See the ``Algorithm`` section below for an explanation of why this is + See the 'Algorithm' section below for an explanation of why this is needed. ---left-only:: +`--left-only`:: Suppress commits that are missing from the first specified range - (or the "left range" when using the `...` format). + (or the "left range" when using the `...` form). ---right-only:: +`--right-only`:: Suppress commits that are missing from the second specified range - (or the "right range" when using the `...` format). + (or the "right range" when using the `...` form). ---diff-merges=:: +`--diff-merges=`:: Instead of ignoring merge commits, generate diffs for them using the corresponding `--diff-merges=` option of linkgit:git-log[1], and include them in the comparison. @@ -93,30 +93,30 @@ have produced. In other words, if a merge commit is the result of a non-conflicting `git merge`, the `remerge` mode will represent it with an empty diff. ---remerge-diff:: +`--remerge-diff`:: Convenience option, equivalent to `--diff-merges=remerge`. ---notes[=]:: ---no-notes:: +`--notes[=]`:: +`--no-notes`:: This flag is passed to the `git log` program (see linkgit:git-log[1]) that generates the patches. - :: +` `:: Compare the commits specified by the two ranges, where - `` is considered an older version of ``. + __ is considered an older version of __. -...:: +`...`:: Equivalent to passing `..` and `..`. - :: +` `:: Equivalent to passing `..` and `..`. - Note that `` does not need to be the exact branch point + Note that __ does not need to be the exact branch point of the branches. Example: after rebasing a branch `my-topic`, `git range-diff my-topic@{u} my-topic@{1} my-topic` would show the differences introduced by the rebase. `git range-diff` also accepts the regular diff options (see -linkgit:git-diff[1]), most notably the `--color=[]` and +linkgit:git-diff[1]), most notably the `--color[=]` and `--no-color` options. These options are used when generating the "diff between patches", i.e. to compare the author, commit message and diff of corresponding old/new commits. There is currently no means to tweak most of the diff --git a/Documentation/git-shortlog.adoc b/Documentation/git-shortlog.adoc index a11b57c1cd7b2d..e067d39b3880a8 100644 --- a/Documentation/git-shortlog.adoc +++ b/Documentation/git-shortlog.adoc @@ -3,63 +3,63 @@ git-shortlog(1) NAME ---- -git-shortlog - Summarize 'git log' output +git-shortlog - Summarize `git log` output SYNOPSIS -------- -[verse] -'git shortlog' [] [] [[--] ...] -git log --pretty=short | 'git shortlog' [] +[synopsis] +git shortlog [] [] [[--] ...] +git log --pretty=short | git shortlog [] DESCRIPTION ----------- -Summarizes 'git log' output in a format suitable for inclusion +Summarizes `git log` output in a format suitable for inclusion in release announcements. Each commit will be grouped by author and title. Additionally, "[PATCH]" will be stripped from the commit description. If no revisions are passed on the command line and either standard input -is not a terminal or there is no current branch, 'git shortlog' will +is not a terminal or there is no current branch, `git shortlog` will output a summary of the log read from standard input, without reference to the current repository. OPTIONS ------- --n:: ---numbered:: +`-n`:: +`--numbered`:: Sort output according to the number of commits per author instead of author alphabetic order. --s:: ---summary:: +`-s`:: +`--summary`:: Suppress commit description and provide a commit count summary only. --e:: ---email:: +`-e`:: +`--email`:: Show the email address of each author. ---format[=]:: +`--format[=]`:: Instead of the commit subject, use some other information to - describe each commit. '' can be any string accepted - by the `--format` option of 'git log', such as '* [%h] %s'. - (See the "PRETTY FORMATS" section of linkgit:git-log[1].) + describe each commit. __ can be any string accepted + by the `--format` option of `git log`, such as '* [%h] %s'. + (See the 'PRETTY FORMATS' section of linkgit:git-log[1].) + Each pretty-printed commit will be rewrapped before it is shown. ---date=:: +`--date=`:: Show dates formatted according to the given date string. (See - the `--date` option in the "Commit Formatting" section of + the `--date` option in the 'Commit Formatting' section of linkgit:git-log[1]). Useful with `--group=format:`. ---group=:: - Group commits based on ``. If no `--group` option is - specified, the default is `author`. `` is one of: +`--group=`:: + Group commits based on __. If no `--group` option is + specified, the default is `author`. __ is one of: + -- - `author`, commits are grouped by author - `committer`, commits are grouped by committer (the same as `-c`) - - `trailer:`, the `` is interpreted as a case-insensitive + - `trailer:`, the __ is interpreted as a case-insensitive commit message trailer (see linkgit:git-interpret-trailers[1]). For example, if your project uses `Reviewed-by` trailers, you might want to see who has been reviewing with @@ -76,7 +76,7 @@ unless the `--email` option is specified. If the value cannot be parsed as an identity, it will be taken literally and completely. - `format:`, any string accepted by the `--format` option of - 'git log'. (See the "PRETTY FORMATS" section of + `git log`. (See the 'PRETTY FORMATS' section of linkgit:git-log[1].) -- + @@ -85,11 +85,11 @@ value (but again, only once per unique value in that commit). For example, `git shortlog --group=author --group=trailer:co-authored-by` counts both authors and co-authors. --c:: ---committer:: +`-c`:: +`--committer`:: This is an alias for `--group=committer`. --w[[,[,]]]:: +`-w[[,[,]]]`:: Linewrap the output by wrapping each line at `width`. The first line of each entry is indented by `indent1` spaces, and the second and subsequent lines are indented by `indent2` spaces. `width`, @@ -98,16 +98,16 @@ counts both authors and co-authors. If width is `0` (zero) then indent the lines of the output without wrapping them. -:: +``:: Show only commits in the specified revision range. When no - is specified, it defaults to `HEAD` (i.e. the + __ is specified, it defaults to `HEAD` (i.e. the whole history leading to the current commit). `origin..HEAD` specifies all the commits reachable from the current commit (i.e. `HEAD`), but not from `origin`. For a complete list of - ways to spell , see the "Specifying Ranges" + ways to spell __, see the 'Specifying Ranges' section of linkgit:gitrevisions[7]. -[--] ...:: +`[--] ...`:: Consider only commits that are enough to explain how the files that match the specified paths came to be. + diff --git a/Documentation/gitprotocol-v2.adoc b/Documentation/gitprotocol-v2.adoc index ac5102b64f99ec..befa697d21c281 100644 --- a/Documentation/gitprotocol-v2.adoc +++ b/Documentation/gitprotocol-v2.adoc @@ -848,6 +848,10 @@ advertised, it can reply with "promisor-remote=" where where `pr-name` is the urlencoded name of a promisor remote the server advertised and the client accepts. +The promisor remotes that the client accepted will be tried before the +other configured promisor remotes when the client attempts to fetch +missing objects. + Note that, everywhere in this document, the ';' and ',' characters MUST be encoded if they appear in `pr-name` or `field-value`. diff --git a/Makefile b/Makefile index cedc234173e377..bd24dd0409f97f 100644 --- a/Makefile +++ b/Makefile @@ -2671,7 +2671,7 @@ git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS) help.sp help.s help.o: command-list.h builtin/bugreport.sp builtin/bugreport.s builtin/bugreport.o: hook-list.h -builtin/hook.sp builtin/hook.s builtin/hook.o: hook-list.h +hook.sp hook.s hook.o: hook-list.h builtin/help.sp builtin/help.s builtin/help.o: config-list.h GIT-PREFIX builtin/help.sp builtin/help.s builtin/help.o: EXTRA_CPPFLAGS = \ diff --git a/builtin/am.c b/builtin/am.c index fe6e087eee9ff5..e9623b8307793f 100644 --- a/builtin/am.c +++ b/builtin/am.c @@ -490,9 +490,11 @@ static int run_applypatch_msg_hook(struct am_state *state) assert(state->msg); - if (!state->no_verify) - ret = run_hooks_l(the_repository, "applypatch-msg", - am_path(state, "final-commit"), NULL); + if (!state->no_verify) { + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; + strvec_push(&opt.args, am_path(state, "final-commit")); + ret = run_hooks_opt(the_repository, "applypatch-msg", &opt); + } if (!ret) { FREE_AND_NULL(state->msg); diff --git a/builtin/checkout.c b/builtin/checkout.c index e031e6188613a6..ac0186a33e559a 100644 --- a/builtin/checkout.c +++ b/builtin/checkout.c @@ -31,6 +31,7 @@ #include "resolve-undo.h" #include "revision.h" #include "setup.h" +#include "strvec.h" #include "submodule.h" #include "symlinks.h" #include "trace2.h" @@ -123,13 +124,19 @@ static void branch_info_release(struct branch_info *info) static int post_checkout_hook(struct commit *old_commit, struct commit *new_commit, int changed) { - return run_hooks_l(the_repository, "post-checkout", - oid_to_hex(old_commit ? &old_commit->object.oid : null_oid(the_hash_algo)), - oid_to_hex(new_commit ? &new_commit->object.oid : null_oid(the_hash_algo)), - changed ? "1" : "0", NULL); - /* "new_commit" can be NULL when checking out from the index before - a commit exists. */ + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; + /* + * "new_commit" can be NULL when checking out from the index before + * a commit exists. + */ + strvec_pushl(&opt.args, + oid_to_hex(old_commit ? &old_commit->object.oid : null_oid(the_hash_algo)), + oid_to_hex(new_commit ? &new_commit->object.oid : null_oid(the_hash_algo)), + changed ? "1" : "0", + NULL); + + return run_hooks_opt(the_repository, "post-checkout", &opt); } static int update_some(const struct object_id *oid, struct strbuf *base, diff --git a/builtin/clone.c b/builtin/clone.c index fba3c9c508bc06..d23b0cafcfec30 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -647,6 +647,7 @@ static int checkout(int submodule_progress, struct tree *tree; struct tree_desc t; int err = 0; + struct run_hooks_opt hook_opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; if (option_no_checkout) return 0; @@ -697,8 +698,9 @@ static int checkout(int submodule_progress, if (write_locked_index(the_repository->index, &lock_file, COMMIT_LOCK)) die(_("unable to write new index file")); - err |= run_hooks_l(the_repository, "post-checkout", oid_to_hex(null_oid(the_hash_algo)), - oid_to_hex(&oid), "1", NULL); + strvec_pushl(&hook_opt.args, oid_to_hex(null_oid(the_hash_algo)), + oid_to_hex(&oid), "1", NULL); + err |= run_hooks_opt(the_repository, "post-checkout", &hook_opt); if (!err && (option_recurse_submodules.nr > 0)) { struct child_process cmd = CHILD_PROCESS_INIT; diff --git a/builtin/hook.c b/builtin/hook.c index c0585587e5e4fa..cceeb3586e5daf 100644 --- a/builtin/hook.c +++ b/builtin/hook.c @@ -4,23 +4,15 @@ #include "environment.h" #include "gettext.h" #include "hook.h" -#include "hook-list.h" #include "parse-options.h" +#include "thread-utils.h" #define BUILTIN_HOOK_RUN_USAGE \ - N_("git hook run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [-- ]") + N_("git hook run [--allow-unknown-hook-name] [--ignore-missing] [--to-stdin=] [(-j|--jobs) ]\n" \ + " [-- ]") #define BUILTIN_HOOK_LIST_USAGE \ N_("git hook list [--allow-unknown-hook-name] [-z] [--show-scope] ") -static int is_known_hook(const char *name) -{ - const char **p; - for (p = hook_name_list; *p; p++) - if (!strcmp(*p, name)) - return 1; - return 0; -} - static const char * const builtin_hook_usage[] = { BUILTIN_HOOK_RUN_USAGE, BUILTIN_HOOK_LIST_USAGE, @@ -96,14 +88,22 @@ static int list(int argc, const char **argv, const char *prefix, const char *name = h->u.configured.friendly_name; const char *scope = show_scope ? config_scope_name(h->u.configured.scope) : NULL; + /* + * Show the most relevant disable reason. Event-level + * takes precedence: if the whole event is off, that + * is what the user needs to know. The per-hook + * "disabled" surfaces once the event is re-enabled. + */ + const char *disability = + h->u.configured.event_disabled ? "event-disabled\t" : + h->u.configured.disabled ? "disabled\t" : + ""; if (scope) - printf("%s\t%s%s%c", scope, - h->u.configured.disabled ? "disabled\t" : "", - name, line_terminator); + printf("%s\t%s%s%c", scope, disability, name, + line_terminator); else - printf("%s%s%c", - h->u.configured.disabled ? "disabled\t" : "", - name, line_terminator); + printf("%s%s%c", disability, name, + line_terminator); break; } default: @@ -124,6 +124,7 @@ static int run(int argc, const char **argv, const char *prefix, struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; int ignore_missing = 0; int allow_unknown = 0; + int jobs = 0; const char *hook_name; struct option run_options[] = { OPT_BOOL(0, "allow-unknown-hook-name", &allow_unknown, @@ -132,6 +133,8 @@ static int run(int argc, const char **argv, const char *prefix, N_("silently ignore missing requested ")), OPT_STRING(0, "to-stdin", &opt.path_to_stdin, N_("path"), N_("file to read into hooks' stdin")), + OPT_INTEGER('j', "jobs", &jobs, + N_("run up to hooks simultaneously (-1 for CPU count)")), OPT_END(), }; int ret; @@ -140,6 +143,15 @@ static int run(int argc, const char **argv, const char *prefix, builtin_hook_run_usage, PARSE_OPT_KEEP_DASHDASH); + if (jobs == -1) + opt.jobs = online_cpus(); + else if (jobs < 0) + die(_("invalid value for -j: %d" + " (use -1 for CPU count or a" + " positive integer)"), jobs); + else + opt.jobs = jobs; + if (!argc) goto usage; diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c index 878aa7f0ed9eb5..f0771590a73ef0 100644 --- a/builtin/receive-pack.c +++ b/builtin/receive-pack.c @@ -1456,7 +1456,8 @@ static const char *push_to_checkout(unsigned char *hash, struct strvec *env, const char *work_tree) { - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; + opt.invoked_hook = invoked_hook; strvec_pushf(env, "GIT_WORK_TREE=%s", absolute_path(work_tree)); diff --git a/builtin/worktree.c b/builtin/worktree.c index 4fd6f7575f9f76..d21c43fde38b5e 100644 --- a/builtin/worktree.c +++ b/builtin/worktree.c @@ -609,7 +609,7 @@ static int add_worktree(const char *path, const char *refname, * is_junk is cleared, but do return appropriate code when hook fails. */ if (!ret && opts->checkout && !opts->orphan) { - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; strvec_pushl(&opt.env, "GIT_DIR", "GIT_WORK_TREE", NULL); strvec_pushl(&opt.args, diff --git a/cache-tree.c b/cache-tree.c index 7881b42aa24c80..2b636eb3f8bddd 100644 --- a/cache-tree.c +++ b/cache-tree.c @@ -238,8 +238,8 @@ int cache_tree_fully_valid(struct cache_tree *it) if (!it) return 0; if (it->entry_count < 0 || - odb_has_object(the_repository->objects, &it->oid, - ODB_HAS_OBJECT_RECHECK_PACKED | ODB_HAS_OBJECT_FETCH_PROMISOR)) + !odb_has_object(the_repository->objects, &it->oid, + ODB_HAS_OBJECT_RECHECK_PACKED | ODB_HAS_OBJECT_FETCH_PROMISOR)) return 0; for (i = 0; i < it->subtree_nr; i++) { if (!cache_tree_fully_valid(it->down[i]->cache_tree)) diff --git a/commit.c b/commit.c index 80d8d078757dbc..4385ae4329e921 100644 --- a/commit.c +++ b/commit.c @@ -1970,7 +1970,7 @@ size_t ignored_log_message_bytes(const char *buf, size_t len) int run_commit_hook(int editor_is_used, const char *index_file, int *invoked_hook, const char *name, ...) { - struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT; + struct run_hooks_opt opt = RUN_HOOKS_OPT_INIT_FORCE_SERIAL; va_list args; const char *arg; diff --git a/config.c b/config.c index 156f2a24fa0027..a1b92fe083cf43 100644 --- a/config.c +++ b/config.c @@ -1212,6 +1212,15 @@ int git_config_int(const char *name, const char *value, return ret; } +unsigned int git_config_uint(const char *name, const char *value, + const struct key_value_info *kvi) +{ + unsigned int ret; + if (!git_parse_uint(value, &ret)) + die_bad_number(name, value, kvi); + return ret; +} + int64_t git_config_int64(const char *name, const char *value, const struct key_value_info *kvi) { @@ -1907,6 +1916,18 @@ int git_configset_get_int(struct config_set *set, const char *key, int *dest) return 1; } +int git_configset_get_uint(struct config_set *set, const char *key, unsigned int *dest) +{ + const char *value; + struct key_value_info kvi; + + if (!git_configset_get_value(set, key, &value, &kvi)) { + *dest = git_config_uint(key, value, &kvi); + return 0; + } else + return 1; +} + int git_configset_get_ulong(struct config_set *set, const char *key, unsigned long *dest) { const char *value; @@ -2356,6 +2377,13 @@ int repo_config_get_int(struct repository *repo, return git_configset_get_int(repo->config, key, dest); } +int repo_config_get_uint(struct repository *repo, + const char *key, unsigned int *dest) +{ + git_config_check_init(repo); + return git_configset_get_uint(repo->config, key, dest); +} + int repo_config_get_ulong(struct repository *repo, const char *key, unsigned long *dest) { diff --git a/config.h b/config.h index ba426a960af9f4..bf47fb3afc61bf 100644 --- a/config.h +++ b/config.h @@ -267,6 +267,12 @@ int git_config_int(const char *, const char *, const struct key_value_info *); int64_t git_config_int64(const char *, const char *, const struct key_value_info *); +/** + * Identical to `git_config_int`, but for unsigned ints. + */ +unsigned int git_config_uint(const char *, const char *, + const struct key_value_info *); + /** * Identical to `git_config_int`, but for unsigned longs. */ @@ -560,6 +566,7 @@ int git_configset_get_value(struct config_set *cs, const char *key, int git_configset_get_string(struct config_set *cs, const char *key, char **dest); int git_configset_get_int(struct config_set *cs, const char *key, int *dest); +int git_configset_get_uint(struct config_set *cs, const char *key, unsigned int *dest); int git_configset_get_ulong(struct config_set *cs, const char *key, unsigned long *dest); int git_configset_get_bool(struct config_set *cs, const char *key, int *dest); int git_configset_get_bool_or_int(struct config_set *cs, const char *key, int *is_bool, int *dest); @@ -650,6 +657,12 @@ int repo_config_get_string_tmp(struct repository *r, */ int repo_config_get_int(struct repository *r, const char *key, int *dest); +/** + * Similar to `repo_config_get_int` but for unsigned ints. + */ +int repo_config_get_uint(struct repository *r, + const char *key, unsigned int *dest); + /** * Similar to `repo_config_get_int` but for unsigned longs. */ diff --git a/hook.c b/hook.c index cc23276d27f035..d10eef4763c679 100644 --- a/hook.c +++ b/hook.c @@ -5,12 +5,23 @@ #include "environment.h" #include "gettext.h" #include "hook.h" +#include "hook-list.h" #include "parse.h" #include "path.h" #include "run-command.h" #include "setup.h" #include "strbuf.h" #include "strmap.h" +#include "thread-utils.h" + +bool is_known_hook(const char *name) +{ + const char **h; + for (h = hook_name_list; *h; h++) + if (!strcmp(*h, name)) + return true; + return false; +} const char *find_hook(struct repository *r, const char *name) { @@ -116,18 +127,27 @@ struct hook_config_cache_entry { char *command; enum config_scope scope; bool disabled; + bool parallel; }; /* * Callback struct to collect all hook.* keys in a single config pass. * commands: friendly-name to command map. * event_hooks: event-name to list of friendly-names map. - * disabled_hooks: set of friendly-names with hook..enabled = false. + * disabled_hooks: set of all names with hook..enabled = false; after + * parsing, names that are not friendly-names become event-level + * disables stored in r->disabled_events. This collects all. + * parallel_hooks: friendly-name to parallel flag. + * event_jobs: event-name to per-event jobs count (stored as uintptr_t, NULL == unset). + * jobs: value of the global hook.jobs key. Defaults to 0 if unset (stored in r->hook_jobs). */ struct hook_all_config_cb { struct strmap commands; struct strmap event_hooks; struct string_list disabled_hooks; + struct strmap parallel_hooks; + struct strmap event_jobs; + unsigned int jobs; }; /* repo_config() callback that collects all hook.* configuration in one pass. */ @@ -143,6 +163,24 @@ static int hook_config_lookup_all(const char *key, const char *value, if (parse_config_key(key, "hook", &name, &name_len, &subkey)) return 0; + /* Handle plain hook. entries that have no hook name component. */ + if (!name) { + if (!strcmp(subkey, "jobs") && value) { + int v; + if (!git_parse_int(value, &v)) + warning(_("hook.jobs must be an integer, ignoring: '%s'"), value); + else if (v == -1) + data->jobs = online_cpus(); + else if (v > 0) + data->jobs = v; + else + warning(_("hook.jobs must be a positive integer" + " or -1, ignoring: '%s'"), + value); + } + return 0; + } + if (!value) return config_error_nonbool(key); @@ -158,8 +196,21 @@ static int hook_config_lookup_all(const char *key, const char *value, strmap_for_each_entry(&data->event_hooks, &iter, e) unsorted_string_list_remove(e->value, hook_name, 0); } else { - struct string_list *hooks = - strmap_get(&data->event_hooks, value); + struct string_list *hooks; + + if (is_known_hook(hook_name)) + die(_("hook friendly-name '%s' collides with " + "a known event name; please choose a " + "different friendly-name"), + hook_name); + + if (!strcmp(hook_name, value)) + warning(_("hook friendly-name '%s' is the " + "same as its event; this may cause " + "ambiguity with hook.%s.enabled"), + hook_name, hook_name); + + hooks = strmap_get(&data->event_hooks, value); if (!hooks) { CALLOC_ARRAY(hooks, 1); @@ -203,6 +254,31 @@ static int hook_config_lookup_all(const char *key, const char *value, default: break; /* ignore unrecognised values */ } + } else if (!strcmp(subkey, "parallel")) { + int v = git_parse_maybe_bool(value); + if (v >= 0) + strmap_put(&data->parallel_hooks, hook_name, + (void *)(uintptr_t)v); + else + warning(_("hook.%s.parallel must be a boolean," + " ignoring: '%s'"), + hook_name, value); + } else if (!strcmp(subkey, "jobs")) { + int v; + if (!git_parse_int(value, &v)) + warning(_("hook.%s.jobs must be an integer," + " ignoring: '%s'"), + hook_name, value); + else if (v == -1) + strmap_put(&data->event_jobs, hook_name, + (void *)(uintptr_t)online_cpus()); + else if (v > 0) + strmap_put(&data->event_jobs, hook_name, + (void *)(uintptr_t)v); + else + warning(_("hook.%s.jobs must be a positive" + " integer or -1, ignoring: '%s'"), + hook_name, value); } free(hook_name); @@ -237,20 +313,78 @@ void hook_cache_clear(struct strmap *cache) strmap_clear(cache, 0); } +/* + * Return true if `name` is a hook friendly-name, i.e. it has at least one of + * .command, .event, or .parallel configured. These are the reliable clues + * that distinguish a friendly-name from an event name. Note: .enabled is + * deliberately excluded because it can appear under both namespaces. + */ +static int is_friendly_name(struct hook_all_config_cb *cb, const char *name) +{ + struct hashmap_iter iter; + struct strmap_entry *e; + + if (strmap_get(&cb->commands, name) || strmap_get(&cb->parallel_hooks, name)) + return 1; + + strmap_for_each_entry(&cb->event_hooks, &iter, e) { + if (unsorted_string_list_lookup(e->value, name)) + return 1; + } + + return 0; +} + +/* Warn if any name in event_jobs is also a hook friendly-name. */ +static void warn_jobs_on_friendly_names(struct hook_all_config_cb *cb_data) +{ + struct hashmap_iter iter; + struct strmap_entry *e; + + strmap_for_each_entry(&cb_data->event_jobs, &iter, e) { + if (is_friendly_name(cb_data, e->key)) + warning(_("hook.%s.jobs is set but '%s' looks like a " + "hook friendly-name, not an event name; " + "hook..jobs uses the event name " + "(e.g. hook.post-receive.jobs), so this " + "setting will be ignored"), e->key, e->key); + } +} + /* Populate `cache` with the complete hook configuration */ static void build_hook_config_map(struct repository *r, struct strmap *cache) { - struct hook_all_config_cb cb_data; + struct hook_all_config_cb cb_data = { 0 }; struct hashmap_iter iter; struct strmap_entry *e; strmap_init(&cb_data.commands); strmap_init(&cb_data.event_hooks); string_list_init_dup(&cb_data.disabled_hooks); + strmap_init(&cb_data.parallel_hooks); + strmap_init(&cb_data.event_jobs); - /* Parse all configs in one run. */ + /* Parse all configs in one run, capturing hook.* including hook.jobs. */ repo_config(r, hook_config_lookup_all, &cb_data); + warn_jobs_on_friendly_names(&cb_data); + + /* + * Populate disabled_events: names in disabled_hooks that are not + * friendly-names are event-level switches (hook..enabled = false). + * Names that are friendly-names are already handled per-hook via the + * hook_config_cache_entry.disabled flag below. + */ + if (r) { + string_list_clear(&r->disabled_events, 0); + string_list_init_dup(&r->disabled_events); + for (size_t i = 0; i < cb_data.disabled_hooks.nr; i++) { + const char *n = cb_data.disabled_hooks.items[i].string; + if (!is_friendly_name(&cb_data, n)) + string_list_append(&r->disabled_events, n); + } + } + /* Construct the cache from parsed configs. */ strmap_for_each_entry(&cb_data.event_hooks, &iter, e) { struct string_list *hook_names = e->value; @@ -266,6 +400,7 @@ static void build_hook_config_map(struct repository *r, struct strmap *cache) struct hook_config_cache_entry *entry; char *command; + bool is_par = !!strmap_get(&cb_data.parallel_hooks, hname); bool is_disabled = !!unsorted_string_list_lookup( &cb_data.disabled_hooks, hname); @@ -286,13 +421,20 @@ static void build_hook_config_map(struct repository *r, struct strmap *cache) entry->command = xstrdup_or_null(command); entry->scope = scope; entry->disabled = is_disabled; + entry->parallel = is_par; string_list_append(hooks, hname)->util = entry; } strmap_put(cache, e->key, hooks); } + if (r) { + r->hook_jobs = cb_data.jobs; + r->event_jobs = cb_data.event_jobs; + } + strmap_clear(&cb_data.commands, 1); + strmap_clear(&cb_data.parallel_hooks, 0); /* values are uintptr_t, not heap ptrs */ string_list_clear(&cb_data.disabled_hooks, 0); strmap_for_each_entry(&cb_data.event_hooks, &iter, e) { string_list_clear(e->value, 0); @@ -344,6 +486,8 @@ static void list_hooks_add_configured(struct repository *r, { struct strmap *cache = get_hook_config_cache(r); struct string_list *configured_hooks = strmap_get(cache, hookname); + bool event_is_disabled = r ? !!unsorted_string_list_lookup(&r->disabled_events, + hookname) : 0; /* Iterate through configured hooks and initialize internal states */ for (size_t i = 0; configured_hooks && i < configured_hooks->nr; i++) { @@ -370,6 +514,8 @@ static void list_hooks_add_configured(struct repository *r, entry->command ? xstrdup(entry->command) : NULL; hook->u.configured.scope = entry->scope; hook->u.configured.disabled = entry->disabled; + hook->u.configured.event_disabled = event_is_disabled; + hook->parallel = entry->parallel; string_list_append(list, friendly_name)->util = hook; } @@ -381,6 +527,8 @@ static void list_hooks_add_configured(struct repository *r, if (!r || !r->gitdir) { hook_cache_clear(cache); free(cache); + if (r) + string_list_clear(&r->disabled_events, 0); } } @@ -412,7 +560,7 @@ int hook_exists(struct repository *r, const char *name) for (size_t i = 0; i < hooks->nr; i++) { struct hook *h = hooks->items[i].util; if (h->kind == HOOK_TRADITIONAL || - !h->u.configured.disabled) { + (!h->u.configured.disabled && !h->u.configured.event_disabled)) { exists = 1; break; } @@ -435,7 +583,8 @@ static int pick_next_hook(struct child_process *cp, if (hook_cb->hook_to_run_index >= hook_list->nr) return 0; h = hook_list->items[hook_cb->hook_to_run_index++].util; - } while (h->kind == HOOK_CONFIGURED && h->u.configured.disabled); + } while (h->kind == HOOK_CONFIGURED && + (h->u.configured.disabled || h->u.configured.event_disabled)); cp->no_stdin = 1; strvec_pushv(&cp->env, hook_cb->options->env.v); @@ -519,21 +668,136 @@ static void run_hooks_opt_clear(struct run_hooks_opt *options) strvec_clear(&options->args); } +/* + * When running in parallel, stdout must be merged into stderr so + * run-command can buffer and de-interleave outputs correctly. This + * applies even to hooks like pre-push that normally keep stdout and + * stderr separate: the user has opted into parallelism, so the output + * stream behavior changes accordingly. + */ +static void merge_output_if_parallel(struct run_hooks_opt *options) +{ + if (options->jobs > 1) + options->stdout_to_stderr = 1; +} + +static void warn_non_parallel_hooks_override(unsigned int jobs, + struct string_list *hook_list) +{ + /* Don't warn for hooks running sequentially. */ + if (jobs == 1) + return; + + for (size_t i = 0; i < hook_list->nr; i++) { + struct hook *h = hook_list->items[i].util; + if (h->kind == HOOK_CONFIGURED && !h->parallel) + warning(_("hook '%s' is not marked as parallel=true, " + "running in parallel anyway due to -j%u"), + h->u.configured.friendly_name, jobs); + } +} + +/* Resolve a hook.jobs config key, handling -1 as online_cpus(). */ +static void resolve_hook_config_jobs(struct repository *r, + const char *key, + unsigned int *jobs) +{ + int v; + + if (repo_config_get_int(r, key, &v)) + return; + + if (v == -1) + *jobs = online_cpus(); + else if (v > 0) + *jobs = v; + else + warning(_("%s must be a positive integer or -1," + " ignoring: %d"), key, v); +} + +/* Determine how many jobs to use for hook execution. */ +static unsigned int get_hook_jobs(struct repository *r, + struct run_hooks_opt *options, + const char *hook_name, + struct string_list *hook_list) +{ + /* + * An explicit job count overrides everything else: this covers both + * FORCE_SERIAL callers (for hooks that must never run in parallel) + * and the -j flag from the CLI. The CLI override is intentional: users + * may want to serialize hooks declared parallel or to parallelize more + * aggressively than the default. + */ + if (options->jobs) + goto cleanup; + + /* + * Use hook.jobs from the already-parsed config cache (in-repo), or + * fallback to a direct config lookup (out-of-repo). + * Default to 1 (serial execution) on failure. + */ + options->jobs = 1; + if (r) { + if (r->gitdir && r->hook_config_cache) { + void *event_jobs; + + if (r->hook_jobs) + options->jobs = r->hook_jobs; + + event_jobs = strmap_get(&r->event_jobs, hook_name); + if (event_jobs) + options->jobs = (unsigned int)(uintptr_t)event_jobs; + } else { + char *key; + + resolve_hook_config_jobs(r, "hook.jobs", &options->jobs); + + key = xstrfmt("hook.%s.jobs", hook_name); + resolve_hook_config_jobs(r, key, &options->jobs); + free(key); + } + } + + /* + * Cap to serial any configured hook not marked as parallel = true. + * This enforces the parallel = false default, even for "traditional" + * hooks from the hookdir which cannot be marked parallel = true. + * The same restriction applies whether jobs came from hook.jobs or + * hook..jobs. + */ + for (size_t i = 0; i < hook_list->nr; i++) { + struct hook *h = hook_list->items[i].util; + if (h->kind == HOOK_CONFIGURED && !h->parallel) { + options->jobs = 1; + break; + } + } + +cleanup: + merge_output_if_parallel(options); + warn_non_parallel_hooks_override(options->jobs, hook_list); + return options->jobs; +} + int run_hooks_opt(struct repository *r, const char *hook_name, struct run_hooks_opt *options) { + struct string_list *hook_list = list_hooks(r, hook_name, options); struct hook_cb_data cb_data = { .rc = 0, .hook_name = hook_name, + .hook_command_list = hook_list, .options = options, }; int ret = 0; + unsigned int jobs = get_hook_jobs(r, options, hook_name, hook_list); const struct run_process_parallel_opts opts = { .tr2_category = "hook", .tr2_label = hook_name, - .processes = options->jobs, - .ungroup = options->jobs == 1, + .processes = jobs, + .ungroup = jobs == 1, .get_next_task = pick_next_hook, .start_failure = notify_start_failure, @@ -549,9 +813,6 @@ int run_hooks_opt(struct repository *r, const char *hook_name, if (options->path_to_stdin && options->feed_pipe) BUG("options path_to_stdin and feed_pipe are mutually exclusive"); - if (!options->jobs) - BUG("run_hooks_opt must be called with options.jobs >= 1"); - /* * Ensure cb_data copy and free functions are either provided together, * or neither one is provided. @@ -562,7 +823,6 @@ int run_hooks_opt(struct repository *r, const char *hook_name, if (options->invoked_hook) *options->invoked_hook = 0; - cb_data.hook_command_list = list_hooks(r, hook_name, options); if (!cb_data.hook_command_list->nr) { if (options->error_if_missing) ret = error("cannot find a hook named %s", hook_name); diff --git a/hook.h b/hook.h index 5c5628dd1f822c..b4372b636ff4de 100644 --- a/hook.h +++ b/hook.h @@ -32,9 +32,17 @@ struct hook { const char *command; enum config_scope scope; bool disabled; + bool event_disabled; } configured; } u; + /** + * Whether this hook may run in parallel with other hooks for the same + * event. Only useful for configured (named) hooks. Traditional hooks + * always default to 0 (serial). Set via `hook..parallel = true`. + */ + bool parallel; + /** * Opaque data pointer used to keep internal state across callback calls. * @@ -72,6 +80,8 @@ struct run_hooks_opt { * * If > 1, output will be buffered and de-interleaved (ungroup=0). * If == 1, output will be real-time (ungroup=1). + * If == 0, the 'hook.jobs' config is used or, if the config is unset, + * defaults to 1 (serial execution). */ unsigned int jobs; @@ -97,8 +107,10 @@ struct run_hooks_opt { * Send the hook's stdout to stderr. * * This is the default behavior for all hooks except pre-push, - * which has separate stdout and stderr streams for backwards - * compatibility reasons. + * which keeps stdout and stderr separate for backwards compatibility. + * When parallel execution is requested (jobs > 1), get_hook_jobs() + * overrides this to 1 for all hooks so run-command can de-interleave + * their outputs correctly. */ unsigned int stdout_to_stderr:1; @@ -152,7 +164,23 @@ struct run_hooks_opt { hook_data_free_fn feed_pipe_cb_data_free; }; +/** + * Default initializer for hooks. Parallelism is opt-in: .jobs = 0 defers to + * the 'hook.jobs' config, falling back to serial (1) if unset. + */ #define RUN_HOOKS_OPT_INIT { \ + .env = STRVEC_INIT, \ + .args = STRVEC_INIT, \ + .stdout_to_stderr = 1, \ + .jobs = 0, \ +} + +/** + * Initializer for hooks that must always run sequentially regardless of + * 'hook.jobs'. Use this when git knows the hook cannot safely be parallelized + * .jobs = 1 is non-overridable. + */ +#define RUN_HOOKS_OPT_INIT_FORCE_SERIAL { \ .env = STRVEC_INIT, \ .args = STRVEC_INIT, \ .stdout_to_stderr = 1, \ @@ -207,6 +235,12 @@ void hook_free(void *p, const char *str); */ void hook_cache_clear(struct strmap *cache); +/** + * Returns true if `name` is a recognized hook event name + * (e.g. "pre-commit", "post-receive"). + */ +bool is_known_hook(const char *name); + /** * Returns the path to the hook file, or NULL if the hook is missing * or disabled. Note that this points to static storage that will be diff --git a/meson.build b/meson.build index 11488623bfd8f8..b28571e08e2b54 100644 --- a/meson.build +++ b/meson.build @@ -563,6 +563,18 @@ libgit_sources += custom_target( env: script_environment, ) +libgit_sources += custom_target( + input: 'Documentation/githooks.adoc', + output: 'hook-list.h', + command: [ + shell, + meson.current_source_dir() + '/tools/generate-hooklist.sh', + meson.current_source_dir(), + '@OUTPUT@', + ], + env: script_environment, +) + builtin_sources = [ 'builtin/add.c', 'builtin/am.c', @@ -739,18 +751,6 @@ builtin_sources += custom_target( env: script_environment, ) -builtin_sources += custom_target( - input: 'Documentation/githooks.adoc', - output: 'hook-list.h', - command: [ - shell, - meson.current_source_dir() + '/tools/generate-hooklist.sh', - meson.current_source_dir(), - '@OUTPUT@', - ], - env: script_environment, -) - # This contains the variables for GIT-BUILD-OPTIONS, which we use to propagate # build options to our tests. build_options_config = configuration_data() diff --git a/parse.c b/parse.c index 48313571aab129..d77f28046a0916 100644 --- a/parse.c +++ b/parse.c @@ -107,6 +107,15 @@ int git_parse_int64(const char *value, int64_t *ret) return 1; } +int git_parse_uint(const char *value, unsigned int *ret) +{ + uintmax_t tmp; + if (!git_parse_unsigned(value, &tmp, maximum_unsigned_value_of_type(unsigned int))) + return 0; + *ret = tmp; + return 1; +} + int git_parse_ulong(const char *value, unsigned long *ret) { uintmax_t tmp; diff --git a/parse.h b/parse.h index ea32de9a91fbfb..a6dd37c4cba273 100644 --- a/parse.h +++ b/parse.h @@ -5,6 +5,7 @@ int git_parse_signed(const char *value, intmax_t *ret, intmax_t max); int git_parse_unsigned(const char *value, uintmax_t *ret, uintmax_t max); int git_parse_ssize_t(const char *, ssize_t *); int git_parse_ulong(const char *, unsigned long *); +int git_parse_uint(const char *value, unsigned int *ret); int git_parse_int(const char *value, int *ret); int git_parse_int64(const char *value, int64_t *ret); int git_parse_double(const char *value, double *ret); diff --git a/promisor-remote.c b/promisor-remote.c index 96fa215b06a924..38fa05054227f6 100644 --- a/promisor-remote.c +++ b/promisor-remote.c @@ -268,11 +268,35 @@ static int remove_fetched_oids(struct repository *repo, return remaining_nr; } +static int try_promisor_remotes(struct repository *repo, + struct object_id **remaining_oids, + int *remaining_nr, int *to_free, + bool accepted_only) +{ + struct promisor_remote *r = repo->promisor_remote_config->promisors; + + for (; r; r = r->next) { + if (accepted_only != r->accepted) + continue; + if (fetch_objects(repo, r->name, *remaining_oids, *remaining_nr) < 0) { + if (*remaining_nr == 1) + continue; + *remaining_nr = remove_fetched_oids(repo, remaining_oids, + *remaining_nr, *to_free); + if (*remaining_nr) { + *to_free = 1; + continue; + } + } + return 1; /* all fetched */ + } + return 0; +} + void promisor_remote_get_direct(struct repository *repo, const struct object_id *oids, int oid_nr) { - struct promisor_remote *r; struct object_id *remaining_oids = (struct object_id *)oids; int remaining_nr = oid_nr; int to_free = 0; @@ -283,19 +307,13 @@ void promisor_remote_get_direct(struct repository *repo, promisor_remote_init(repo); - for (r = repo->promisor_remote_config->promisors; r; r = r->next) { - if (fetch_objects(repo, r->name, remaining_oids, remaining_nr) < 0) { - if (remaining_nr == 1) - continue; - remaining_nr = remove_fetched_oids(repo, &remaining_oids, - remaining_nr, to_free); - if (remaining_nr) { - to_free = 1; - continue; - } - } + /* Try accepted remotes first (those the server told us to use) */ + if (try_promisor_remotes(repo, &remaining_oids, &remaining_nr, + &to_free, true)) + goto all_fetched; + if (try_promisor_remotes(repo, &remaining_oids, &remaining_nr, + &to_free, false)) goto all_fetched; - } for (i = 0; i < remaining_nr; i++) { if (is_promisor_object(repo, &remaining_oids[i])) @@ -557,6 +575,12 @@ enum accept_promisor { ACCEPT_ALL }; +/* + * Check if a specific field and its advertised value match the local + * configuration of a given promisor remote. + * + * Returns 1 if they match, 0 otherwise. + */ static int match_field_against_config(const char *field, const char *value, struct promisor_info *config_info) { @@ -568,9 +592,18 @@ static int match_field_against_config(const char *field, const char *value, return 0; } +/* + * Check that the advertised fields match the local configuration. + * + * When 'config_entry' is NULL (ACCEPT_ALL mode), every checked field + * must match at least one remote in 'config_info'. + * + * When 'config_entry' points to a specific remote's config, the + * checked fields are compared against that single remote only. + */ static int all_fields_match(struct promisor_info *advertised, struct string_list *config_info, - int in_list) + struct promisor_info *config_entry) { struct string_list *fields = fields_checked(); struct string_list_item *item_checked; @@ -579,7 +612,6 @@ static int all_fields_match(struct promisor_info *advertised, int match = 0; const char *field = item_checked->string; const char *value = NULL; - struct string_list_item *item; if (!strcasecmp(field, promisor_field_filter)) value = advertised->filter; @@ -589,7 +621,11 @@ static int all_fields_match(struct promisor_info *advertised, if (!value) return 0; - if (in_list) { + if (config_entry) { + match = match_field_against_config(field, value, + config_entry); + } else { + struct string_list_item *item; for_each_string_list_item(item, config_info) { struct promisor_info *p = item->util; if (match_field_against_config(field, value, p)) { @@ -597,12 +633,6 @@ static int all_fields_match(struct promisor_info *advertised, break; } } - } else { - item = string_list_lookup(config_info, advertised->name); - if (item) { - struct promisor_info *p = item->util; - match = match_field_against_config(field, value, p); - } } if (!match) @@ -612,6 +642,14 @@ static int all_fields_match(struct promisor_info *advertised, return 1; } +static bool has_control_char(const char *s) +{ + for (const char *c = s; *c; c++) + if (iscntrl(*c)) + return true; + return false; +} + static int should_accept_remote(enum accept_promisor accept, struct promisor_info *advertised, struct string_list *config_info) @@ -621,8 +659,13 @@ static int should_accept_remote(enum accept_promisor accept, const char *remote_name = advertised->name; const char *remote_url = advertised->url; + if (!remote_url || !*remote_url) + BUG("no or empty URL advertised for remote '%s'; " + "this remote should have been rejected earlier", + remote_name); + if (accept == ACCEPT_ALL) - return all_fields_match(advertised, config_info, 1); + return all_fields_match(advertised, config_info, NULL); /* Get config info for that promisor remote */ item = string_list_lookup(config_info, remote_name); @@ -634,23 +677,19 @@ static int should_accept_remote(enum accept_promisor accept, p = item->util; if (accept == ACCEPT_KNOWN_NAME) - return all_fields_match(advertised, config_info, 0); + return all_fields_match(advertised, config_info, p); if (accept != ACCEPT_KNOWN_URL) BUG("Unhandled 'enum accept_promisor' value '%d'", accept); - if (!remote_url || !*remote_url) { - warning(_("no or empty URL advertised for remote '%s'"), remote_name); + if (strcmp(p->url, remote_url)) { + warning(_("known remote named '%s' but with URL '%s' instead of '%s', " + "ignoring this remote"), + remote_name, p->url, remote_url); return 0; } - if (!strcmp(p->url, remote_url)) - return all_fields_match(advertised, config_info, 0); - - warning(_("known remote named '%s' but with URL '%s' instead of '%s'"), - remote_name, p->url, remote_url); - - return 0; + return all_fields_match(advertised, config_info, p); } static int skip_field_name_prefix(const char *elem, const char *field_name, const char **value) @@ -691,9 +730,9 @@ static struct promisor_info *parse_one_advertised_remote(const char *remote_info string_list_clear(&elem_list, 0); - if (!info->name || !info->url) { - warning(_("server advertised a promisor remote without a name or URL: %s"), - remote_info); + if (!info->name || !*info->name || !info->url || !*info->url) { + warning(_("server advertised a promisor remote without a name or URL: '%s', " + "ignoring this remote"), remote_info); promisor_info_free(info); return NULL; } @@ -741,18 +780,14 @@ static bool valid_filter(const char *filter, const char *remote_name) return !res; } -/* Check that a token doesn't contain any control character */ static bool valid_token(const char *token, const char *remote_name) { - const char *c = token; - - for (; *c; c++) - if (iscntrl(*c)) { - warning(_("invalid token '%s' for remote '%s' " - "will not be stored"), - token, remote_name); - return false; - } + if (has_control_char(token)) { + warning(_("invalid token '%s' for remote '%s' " + "will not be stored"), + token, remote_name); + return false; + } return true; } @@ -827,20 +862,12 @@ static bool promisor_store_advertised_fields(struct promisor_info *advertised, return reload_config; } -static void filter_promisor_remote(struct repository *repo, - struct strvec *accepted, - const char *info) +static enum accept_promisor accept_from_server(struct repository *repo) { const char *accept_str; enum accept_promisor accept = ACCEPT_NONE; - struct string_list config_info = STRING_LIST_INIT_NODUP; - struct string_list remote_info = STRING_LIST_INIT_DUP; - struct store_info *store_info = NULL; - struct string_list_item *item; - bool reload_config = false; - struct string_list accepted_filters = STRING_LIST_INIT_DUP; - if (!repo_config_get_string_tmp(the_repository, "promisor.acceptfromserver", &accept_str)) { + if (!repo_config_get_string_tmp(repo, "promisor.acceptfromserver", &accept_str)) { if (!*accept_str || !strcasecmp("None", accept_str)) accept = ACCEPT_NONE; else if (!strcasecmp("KnownUrl", accept_str)) @@ -854,6 +881,20 @@ static void filter_promisor_remote(struct repository *repo, accept_str, "promisor.acceptfromserver"); } + return accept; +} + +static void filter_promisor_remote(struct repository *repo, + struct string_list *accepted_remotes, + const char *info) +{ + struct string_list config_info = STRING_LIST_INIT_NODUP; + struct string_list remote_info = STRING_LIST_INIT_DUP; + struct store_info *store_info = NULL; + struct string_list_item *item; + bool reload_config = false; + enum accept_promisor accept = accept_from_server(repo); + if (accept == ACCEPT_NONE) return; @@ -880,17 +921,10 @@ static void filter_promisor_remote(struct repository *repo, if (promisor_store_advertised_fields(advertised, store_info)) reload_config = true; - strvec_push(accepted, advertised->name); - - /* Capture advertised filters for accepted remotes */ - if (advertised->filter) { - struct string_list_item *i; - i = string_list_append(&accepted_filters, advertised->name); - i->util = xstrdup(advertised->filter); - } + string_list_append(accepted_remotes, advertised->name)->util = advertised; + } else { + promisor_info_free(advertised); } - - promisor_info_free(advertised); } promisor_info_list_clear(&config_info); @@ -900,39 +934,36 @@ static void filter_promisor_remote(struct repository *repo, if (reload_config) repo_promisor_remote_reinit(repo); - /* Apply accepted remote filters to the stable repo state */ - for_each_string_list_item(item, &accepted_filters) { - struct promisor_remote *r = repo_promisor_remote_find(repo, item->string); - if (r) { - free(r->advertised_filter); - r->advertised_filter = item->util; - item->util = NULL; - } - } + /* Apply accepted remotes to the stable repo state */ + for_each_string_list_item(item, accepted_remotes) { + struct promisor_info *info = item->util; + struct promisor_remote *r = repo_promisor_remote_find(repo, info->name); - string_list_clear(&accepted_filters, 1); - - /* Mark the remotes as accepted in the repository state */ - for (size_t i = 0; i < accepted->nr; i++) { - struct promisor_remote *r = repo_promisor_remote_find(repo, accepted->v[i]); - if (r) + if (r) { r->accepted = 1; + if (info->filter) { + free(r->advertised_filter); + r->advertised_filter = xstrdup(info->filter); + } + } } } void promisor_remote_reply(const char *info, char **accepted_out) { - struct strvec accepted = STRVEC_INIT; + struct string_list accepted_remotes = STRING_LIST_INIT_NODUP; - filter_promisor_remote(the_repository, &accepted, info); + filter_promisor_remote(the_repository, &accepted_remotes, info); if (accepted_out) { - if (accepted.nr) { + if (accepted_remotes.nr) { struct strbuf reply = STRBUF_INIT; - for (size_t i = 0; i < accepted.nr; i++) { - if (i) + struct string_list_item *item; + + for_each_string_list_item(item, &accepted_remotes) { + if (reply.len) strbuf_addch(&reply, ';'); - strbuf_addstr_urlencode(&reply, accepted.v[i], allow_unsanitized); + strbuf_addstr_urlencode(&reply, item->string, allow_unsanitized); } *accepted_out = strbuf_detach(&reply, NULL); } else { @@ -940,7 +971,7 @@ void promisor_remote_reply(const char *info, char **accepted_out) } } - strvec_clear(&accepted); + promisor_info_list_clear(&accepted_remotes); } void mark_promisor_remotes_as_accepted(struct repository *r, const char *remotes) diff --git a/refs.c b/refs.c index bfcb9c7ac3d38c..844785219d5826 100644 --- a/refs.c +++ b/refs.c @@ -989,7 +989,7 @@ enum ref_worktree_type parse_worktree_ref(const char *maybe_worktree_ref, return REF_WORKTREE_SHARED; } -long get_files_ref_lock_timeout_ms(void) +long get_files_ref_lock_timeout_ms(struct repository *repo) { static int configured = 0; @@ -997,7 +997,7 @@ long get_files_ref_lock_timeout_ms(void) static int timeout_ms = 100; if (!configured) { - repo_config_get_int(the_repository, "core.filesreflocktimeout", &timeout_ms); + repo_config_get_int(repo, "core.filesreflocktimeout", &timeout_ms); configured = 1; } @@ -1472,7 +1472,7 @@ int ref_transaction_create(struct ref_transaction *transaction, return 1; } return ref_transaction_update(transaction, refname, new_oid, - null_oid(the_hash_algo), new_target, NULL, flags, + null_oid(transaction->ref_store->repo->hash_algo), new_target, NULL, flags, msg, err); } @@ -1491,7 +1491,7 @@ int ref_transaction_delete(struct ref_transaction *transaction, if (old_target && !(flags & REF_NO_DEREF)) BUG("delete cannot operate on symrefs with deref mode"); return ref_transaction_update(transaction, refname, - null_oid(the_hash_algo), old_oid, + null_oid(transaction->ref_store->repo->hash_algo), old_oid, NULL, old_target, flags, msg, err); } @@ -2379,7 +2379,7 @@ struct ref_store *repo_get_submodule_ref_store(struct repository *repo, subrepo = xmalloc(sizeof(*subrepo)); if (repo_submodule_init(subrepo, repo, submodule, - null_oid(the_hash_algo))) { + null_oid(repo->hash_algo))) { free(subrepo); goto done; } @@ -2571,14 +2571,14 @@ static int transaction_hook_feed_stdin(int hook_stdin_fd, void *pp_cb, void *pp_ strbuf_reset(buf); if (!(update->flags & REF_HAVE_OLD)) - strbuf_addf(buf, "%s ", oid_to_hex(null_oid(the_hash_algo))); + strbuf_addf(buf, "%s ", oid_to_hex(null_oid(transaction->ref_store->repo->hash_algo))); else if (update->old_target) strbuf_addf(buf, "ref:%s ", update->old_target); else strbuf_addf(buf, "%s ", oid_to_hex(&update->old_oid)); if (!(update->flags & REF_HAVE_NEW)) - strbuf_addf(buf, "%s ", oid_to_hex(null_oid(the_hash_algo))); + strbuf_addf(buf, "%s ", oid_to_hex(null_oid(transaction->ref_store->repo->hash_algo))); else if (update->new_target) strbuf_addf(buf, "ref:%s ", update->new_target); else @@ -3146,6 +3146,7 @@ struct migration_data { static int migrate_one_ref(const struct reference *ref, void *cb_data) { struct migration_data *data = cb_data; + const struct git_hash_algo *hash_algo = data->transaction->ref_store->repo->hash_algo; struct strbuf symref_target = STRBUF_INIT; int ret; @@ -3154,7 +3155,7 @@ static int migrate_one_ref(const struct reference *ref, void *cb_data) if (ret < 0) goto done; - ret = ref_transaction_update(data->transaction, ref->name, NULL, null_oid(the_hash_algo), + ret = ref_transaction_update(data->transaction, ref->name, NULL, null_oid(hash_algo), symref_target.buf, NULL, REF_SKIP_CREATE_REFLOG | REF_NO_DEREF, NULL, data->errbuf); if (ret < 0) diff --git a/refs/files-backend.c b/refs/files-backend.c index b3b0c25f84e503..bae37908454430 100644 --- a/refs/files-backend.c +++ b/refs/files-backend.c @@ -792,7 +792,7 @@ static enum ref_transaction_error lock_raw_ref(struct files_ref_store *refs, if (hold_lock_file_for_update_timeout( &lock->lk, ref_file.buf, LOCK_NO_DEREF, - get_files_ref_lock_timeout_ms()) < 0) { + get_files_ref_lock_timeout_ms(transaction->ref_store->repo)) < 0) { int myerr = errno; errno = 0; if (myerr == ENOENT && --attempts_remaining > 0) { @@ -1190,13 +1190,17 @@ static int remove_empty_directories(struct strbuf *path) return remove_dir_recursively(path, REMOVE_DIR_EMPTY_ONLY); } +struct create_reflock_cb { + struct lock_file *lk; + struct repository *repo; +}; + static int create_reflock(const char *path, void *cb) { - struct lock_file *lk = cb; - + struct create_reflock_cb *data = cb; return hold_lock_file_for_update_timeout( - lk, path, LOCK_NO_DEREF, - get_files_ref_lock_timeout_ms()) < 0 ? -1 : 0; + data->lk, path, LOCK_NO_DEREF, + get_files_ref_lock_timeout_ms(data->repo)) < 0 ? -1 : 0; } /* @@ -1208,6 +1212,7 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs, { struct strbuf ref_file = STRBUF_INIT; struct ref_lock *lock; + struct create_reflock_cb cb_data; files_assert_main_repository(refs, "lock_ref_oid_basic"); assert(err); @@ -1229,8 +1234,10 @@ static struct ref_lock *lock_ref_oid_basic(struct files_ref_store *refs, lock->ref_name = xstrdup(refname); lock->count = 1; + cb_data.lk = &lock->lk; + cb_data.repo = refs->base.repo; - if (raceproof_create_file(ref_file.buf, create_reflock, &lock->lk)) { + if (raceproof_create_file(ref_file.buf, create_reflock, &cb_data)) { unable_to_lock_message(ref_file.buf, errno, err); goto error_return; } diff --git a/refs/refs-internal.h b/refs/refs-internal.h index d79e35fd269a6c..e4cfd9e19ee74f 100644 --- a/refs/refs-internal.h +++ b/refs/refs-internal.h @@ -43,7 +43,7 @@ struct ref_transaction; * Return the length of time to retry acquiring a loose reference lock * before giving up, in milliseconds: */ -long get_files_ref_lock_timeout_ms(void); +long get_files_ref_lock_timeout_ms(struct repository *repo); /* * Return true iff refname is minimally safe. "Safe" here means that diff --git a/refs/reftable-backend.c b/refs/reftable-backend.c index daea30a5b4cad9..4921b456ffa7a2 100644 --- a/refs/reftable-backend.c +++ b/refs/reftable-backend.c @@ -399,12 +399,12 @@ static struct ref_store *reftable_be_init(struct repository *repo, default: BUG("unknown hash algorithm %d", repo->hash_algo->format_id); } - refs->write_options.default_permissions = calc_shared_perm(the_repository, 0666 & ~mask); + refs->write_options.default_permissions = calc_shared_perm(repo, 0666 & ~mask); refs->write_options.disable_auto_compact = !git_env_bool("GIT_TEST_REFTABLE_AUTOCOMPACTION", 1); refs->write_options.lock_timeout_ms = 100; - repo_config(the_repository, reftable_be_config, &refs->write_options); + repo_config(repo, reftable_be_config, &refs->write_options); /* * It is somewhat unfortunate that we have to mirror the default block @@ -486,7 +486,7 @@ static int reftable_be_create_on_disk(struct ref_store *ref_store, struct strbuf sb = STRBUF_INIT; strbuf_addf(&sb, "%s/reftable", refs->base.gitdir); - safe_create_dir(the_repository, sb.buf, 1); + safe_create_dir(ref_store->repo, sb.buf, 1); strbuf_reset(&sb); strbuf_release(&sb); diff --git a/repository.c b/repository.c index 9e5537f53961ed..db57b8308b94e7 100644 --- a/repository.c +++ b/repository.c @@ -323,6 +323,7 @@ int repo_init(struct repository *repo, return 0; error: + clear_repository_format(&format); repo_clear(repo); return -1; } @@ -425,6 +426,8 @@ void repo_clear(struct repository *repo) hook_cache_clear(repo->hook_config_cache); FREE_AND_NULL(repo->hook_config_cache); } + strmap_clear(&repo->event_jobs, 0); /* values are uintptr_t, not heap ptrs */ + string_list_clear(&repo->disabled_events, 0); if (repo->promisor_remote_config) { promisor_remote_clear(repo->promisor_remote_config); diff --git a/repository.h b/repository.h index 078059a6e02b10..4969d8b8ebed60 100644 --- a/repository.h +++ b/repository.h @@ -2,6 +2,7 @@ #define REPOSITORY_H #include "strmap.h" +#include "string-list.h" #include "repo-settings.h" #include "environment.h" @@ -172,6 +173,15 @@ struct repository { */ struct strmap *hook_config_cache; + /* Cached value of hook.jobs config (0 if unset, defaults to serial). */ + unsigned int hook_jobs; + + /* Cached map of event-name -> jobs count (as uintptr_t) from hook..jobs. */ + struct strmap event_jobs; + + /* Cached list of event names with hook..enabled = false. */ + struct string_list disabled_events; + /* Configurations related to promisor remotes. */ char *repository_format_partial_clone; struct promisor_remote_config *promisor_remote_config; diff --git a/sideband.c b/sideband.c index ea7c25211ef7e1..04282a568edd90 100644 --- a/sideband.c +++ b/sideband.c @@ -10,6 +10,7 @@ #include "help.h" #include "pkt-line.h" #include "write-or-die.h" +#include "urlmatch.h" struct keyword_entry { /* @@ -26,6 +27,85 @@ static struct keyword_entry keywords[] = { { "error", GIT_COLOR_BOLD_RED }, }; +static enum { + ALLOW_CONTROL_SEQUENCES_UNSET = -1, + ALLOW_NO_CONTROL_CHARACTERS = 0, + ALLOW_ANSI_COLOR_SEQUENCES = 1<<0, + ALLOW_ANSI_CURSOR_MOVEMENTS = 1<<1, + ALLOW_ANSI_ERASE = 1<<2, + ALLOW_ALL_CONTROL_CHARACTERS = 1<<3, + ALLOW_DEFAULT_ANSI_SEQUENCES = ALLOW_ANSI_COLOR_SEQUENCES +} allow_control_characters = ALLOW_CONTROL_SEQUENCES_UNSET; + +static inline int skip_prefix_in_csv(const char *value, const char *prefix, + const char **out) +{ + if (!skip_prefix(value, prefix, &value) || + (*value && *value != ',')) + return 0; + *out = value + !!*value; + return 1; +} + +int sideband_allow_control_characters_config(const char *var, const char *value) +{ + switch (git_parse_maybe_bool(value)) { + case 0: + allow_control_characters = ALLOW_NO_CONTROL_CHARACTERS; + return 0; + case 1: + allow_control_characters = ALLOW_ALL_CONTROL_CHARACTERS; + return 0; + default: + break; + } + + allow_control_characters = ALLOW_NO_CONTROL_CHARACTERS; + while (*value) { + if (skip_prefix_in_csv(value, "color", &value)) + allow_control_characters |= ALLOW_ANSI_COLOR_SEQUENCES; + else if (skip_prefix_in_csv(value, "cursor", &value)) + allow_control_characters |= ALLOW_ANSI_CURSOR_MOVEMENTS; + else if (skip_prefix_in_csv(value, "erase", &value)) + allow_control_characters |= ALLOW_ANSI_ERASE; + else if (skip_prefix_in_csv(value, "true", &value)) + allow_control_characters = ALLOW_ALL_CONTROL_CHARACTERS; + else if (skip_prefix_in_csv(value, "false", &value)) + allow_control_characters = ALLOW_NO_CONTROL_CHARACTERS; + else + warning(_("unrecognized value for '%s': '%s'"), var, value); + } + return 0; +} + +static int sideband_config_callback(const char *var, const char *value, + const struct config_context *ctx UNUSED, + void *data UNUSED) +{ + if (!strcmp(var, "sideband.allowcontrolcharacters")) + return sideband_allow_control_characters_config(var, value); + + return 0; +} + +void sideband_apply_url_config(const char *url) +{ + struct urlmatch_config config = URLMATCH_CONFIG_INIT; + char *normalized_url; + + if (!url) + BUG("must not call sideband_apply_url_config(NULL)"); + + config.section = "sideband"; + config.collect_fn = sideband_config_callback; + + normalized_url = url_normalize(url, &config.url); + repo_config(the_repository, urlmatch_config_entry, &config); + free(normalized_url); + string_list_clear(&config.vars, 1); + urlmatch_config_release(&config); +} + /* Returns a color setting (GIT_COLOR_NEVER, etc). */ static enum git_colorbool use_sideband_colors(void) { @@ -39,6 +119,14 @@ static enum git_colorbool use_sideband_colors(void) if (use_sideband_colors_cached != GIT_COLOR_UNKNOWN) return use_sideband_colors_cached; + if (allow_control_characters == ALLOW_CONTROL_SEQUENCES_UNSET) { + if (!repo_config_get_value(the_repository, "sideband.allowcontrolcharacters", &value)) + sideband_allow_control_characters_config("sideband.allowcontrolcharacters", value); + + if (allow_control_characters == ALLOW_CONTROL_SEQUENCES_UNSET) + allow_control_characters = ALLOW_DEFAULT_ANSI_SEQUENCES; + } + if (!repo_config_get_string_tmp(the_repository, key, &value)) use_sideband_colors_cached = git_config_colorbool(key, value); else if (!repo_config_get_string_tmp(the_repository, "color.ui", &value)) @@ -66,6 +154,93 @@ void list_config_color_sideband_slots(struct string_list *list, const char *pref list_config_item(list, prefix, keywords[i].keyword); } +static int handle_ansi_sequence(struct strbuf *dest, const char *src, int n) +{ + int i; + + /* + * Valid ANSI color sequences are of the form + * + * ESC [ [ [; ]*] m + * + * These are part of the Select Graphic Rendition sequences which + * contain more than just color sequences, for more details see + * https://en.wikipedia.org/wiki/ANSI_escape_code#SGR. + * + * The cursor movement sequences are: + * + * ESC [ n A - Cursor up n lines (CUU) + * ESC [ n B - Cursor down n lines (CUD) + * ESC [ n C - Cursor forward n columns (CUF) + * ESC [ n D - Cursor back n columns (CUB) + * ESC [ n E - Cursor next line, beginning (CNL) + * ESC [ n F - Cursor previous line, beginning (CPL) + * ESC [ n G - Cursor to column n (CHA) + * ESC [ n ; m H - Cursor position (row n, col m) (CUP) + * ESC [ n ; m f - Same as H (HVP) + * + * The sequences to erase characters are: + * + * + * ESC [ 0 J - Clear from cursor to end of screen (ED) + * ESC [ 1 J - Clear from cursor to beginning of screen (ED) + * ESC [ 2 J - Clear entire screen (ED) + * ESC [ 3 J - Clear entire screen + scrollback (ED) - xterm extension + * ESC [ 0 K - Clear from cursor to end of line (EL) + * ESC [ 1 K - Clear from cursor to beginning of line (EL) + * ESC [ 2 K - Clear entire line (EL) + * ESC [ n M - Delete n lines (DL) + * ESC [ n P - Delete n characters (DCH) + * ESC [ n X - Erase n characters (ECH) + * + * For a comprehensive list of common ANSI Escape sequences, see + * https://www.xfree86.org/current/ctlseqs.html + */ + + if (n < 3 || src[0] != '\x1b' || src[1] != '[') + return 0; + + for (i = 2; i < n; i++) { + if (((allow_control_characters & ALLOW_ANSI_COLOR_SEQUENCES) && + src[i] == 'm') || + ((allow_control_characters & ALLOW_ANSI_CURSOR_MOVEMENTS) && + strchr("ABCDEFGHf", src[i])) || + ((allow_control_characters & ALLOW_ANSI_ERASE) && + strchr("JKMPX", src[i]))) { + strbuf_add(dest, src, i + 1); + return i; + } + if (!isdigit(src[i]) && src[i] != ';') + break; + } + + return 0; +} + +static void strbuf_add_sanitized(struct strbuf *dest, const char *src, int n) +{ + int i; + + if ((allow_control_characters & ALLOW_ALL_CONTROL_CHARACTERS)) { + strbuf_add(dest, src, n); + return; + } + + strbuf_grow(dest, n); + for (; n && *src; src++, n--) { + if (!iscntrl(*src) || *src == '\t' || *src == '\n') { + strbuf_addch(dest, *src); + } else if (allow_control_characters != ALLOW_NO_CONTROL_CHARACTERS && + (i = handle_ansi_sequence(dest, src, n))) { + src += i; + n -= i; + } else { + strbuf_addch(dest, '^'); + strbuf_addch(dest, *src == 0x7f ? '?' : 0x40 + *src); + } + } +} + /* * Optionally highlight one keyword in remote output if it appears at the start * of the line. This should be called for a single line only, which is @@ -81,7 +256,7 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n) int i; if (!want_color_stderr(use_sideband_colors())) { - strbuf_add(dest, src, n); + strbuf_add_sanitized(dest, src, n); return; } @@ -114,7 +289,7 @@ static void maybe_colorize_sideband(struct strbuf *dest, const char *src, int n) } } - strbuf_add(dest, src, n); + strbuf_add_sanitized(dest, src, n); } diff --git a/sideband.h b/sideband.h index 5a25331be55d30..d15fa4015fa0a3 100644 --- a/sideband.h +++ b/sideband.h @@ -30,4 +30,18 @@ int demultiplex_sideband(const char *me, int status, void send_sideband(int fd, int band, const char *data, ssize_t sz, int packet_max); +/* + * Apply sideband configuration for the given URL. This should be called + * when a transport is created to allow URL-specific configuration of + * sideband behavior (e.g., sideband..allowControlCharacters). + */ +void sideband_apply_url_config(const char *url); + +/* + * Parse and set the sideband allow control characters configuration. + * The var parameter should be the key name (without section prefix). + * Returns 0 if the variable was recognized and handled, non-zero otherwise. + */ +int sideband_allow_control_characters_config(const char *var, const char *value); + #endif diff --git a/t/t0090-cache-tree.sh b/t/t0090-cache-tree.sh index d901588294668c..0964718d7f33f5 100755 --- a/t/t0090-cache-tree.sh +++ b/t/t0090-cache-tree.sh @@ -278,4 +278,12 @@ test_expect_success 'switching trees does not invalidate shared index' ' ) ' +test_expect_success 'cache-tree is used by write-tree when valid' ' + test_commit use-valid && + + # write-tree with a valid cache-tree should skip cache_tree_update + GIT_TRACE2_PERF="$(pwd)/trace.output" git write-tree && + test_grep ! region_enter.*cache_tree.*update trace.output +' + test_done diff --git a/t/t1800-hook.sh b/t/t1800-hook.sh index 33decc66c0ea8d..0132e772e472e2 100755 --- a/t/t1800-hook.sh +++ b/t/t1800-hook.sh @@ -21,6 +21,57 @@ setup_hookdir () { test_when_finished rm -rf .git/hooks } +# write_sentinel_hook [sentinel] +# +# Writes a hook that marks itself as started, sleeps for a few seconds, then +# marks itself done. The sleep must be long enough that sentinel_detector can +# observe .started before .done appears when both hooks +# run concurrently in parallel mode. +write_sentinel_hook () { + sentinel="${2:-sentinel}" + write_script "$1" <<-EOF + touch ${sentinel}.started && + sleep 2 && + touch ${sentinel}.done + EOF +} + +# sentinel_detector +# +# Returns a shell command string suitable for use as hook..command. +# The detector must be registered after the sentinel: +# 1. In serial mode, the sentinel has completed (and .done exists) +# before the detector starts. +# 2. In parallel mode, both run concurrently so .done has not appeared +# yet and the detector just sees .started. +# +# At start, poll until .started exists to absorb startup jitter, then +# write to : +# 1. 'serial' if .done exists (sentinel finished before we started), +# 2. 'parallel' if only .started exists (sentinel still running), +# 3. 'timeout' if .started never appeared. +# +# The command ends with ':' so when git appends "$@" for hooks that receive +# positional arguments (e.g. pre-push), the result ': "$@"' is valid shell +# rather than a syntax error 'fi "$@"'. +sentinel_detector () { + cat <<-EOF + i=0 + while ! test -f ${1}.started && test \$i -lt 10; do + sleep 1 + i=\$((i+1)) + done + if test -f ${1}.done; then + echo serial >${2} + elif test -f ${1}.started; then + echo parallel >${2} + else + echo timeout >${2} + fi + : + EOF +} + test_expect_success 'git hook usage' ' test_expect_code 129 git hook && test_expect_code 129 git hook run && @@ -217,10 +268,20 @@ test_expect_success 'git -c core.hooksPath= hook run' ' ' test_hook_tty () { - cat >expect <<-\EOF - STDOUT TTY - STDERR TTY - EOF + expect_tty=$1 + shift + + if test "$expect_tty" != "no_tty"; then + cat >expect <<-\EOF + STDOUT TTY + STDERR TTY + EOF + else + cat >expect <<-\EOF + STDOUT NO TTY + STDERR NO TTY + EOF + fi test_when_finished "rm -rf repo" && git init repo && @@ -238,12 +299,21 @@ test_hook_tty () { test_cmp expect repo/actual } -test_expect_success TTY 'git hook run: stdout and stderr are connected to a TTY' ' - test_hook_tty hook run pre-commit +test_expect_success TTY 'git hook run -j1: stdout and stderr are connected to a TTY' ' + # hooks running sequentially (-j1) are always connected to the tty for + # optimum real-time performance. + test_hook_tty tty hook run -j1 pre-commit +' + +test_expect_success TTY 'git hook run -jN: stdout and stderr are not connected to a TTY' ' + # Hooks are not connected to the tty when run in parallel, instead they + # output to a pipe through which run-command collects and de-interlaces + # their outputs, which then gets passed either to the tty or a sideband. + test_hook_tty no_tty hook run -j2 pre-commit ' test_expect_success TTY 'git commit: stdout and stderr are connected to a TTY' ' - test_hook_tty commit -m"B.new" + test_hook_tty tty commit -m"B.new" ' test_expect_success 'git hook list orders by config order' ' @@ -658,4 +728,504 @@ test_expect_success 'server push-to-checkout hook expects stdout redirected to s check_stdout_merged_to_stderr push-to-checkout ' +test_expect_success 'parallel hook output is not interleaved' ' + test_when_finished "rm -rf .git/hooks" && + + write_script .git/hooks/test-hook <<-EOF && + echo "Hook 1 Start" + sleep 1 + echo "Hook 1 End" + EOF + + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "echo \"Hook 2 Start\"; sleep 2; echo \"Hook 2 End\"" && + test_config hook.hook-2.parallel true && + test_config hook.hook-3.event test-hook && + test_config hook.hook-3.command \ + "echo \"Hook 3 Start\"; sleep 3; echo \"Hook 3 End\"" && + test_config hook.hook-3.parallel true && + + git hook run --allow-unknown-hook-name -j3 test-hook >out 2>err.parallel && + + # Verify Hook 1 output is grouped + sed -n "/Hook 1 Start/,/Hook 1 End/p" err.parallel >hook1_out && + test_line_count = 2 hook1_out && + + # Verify Hook 2 output is grouped + sed -n "/Hook 2 Start/,/Hook 2 End/p" err.parallel >hook2_out && + test_line_count = 2 hook2_out && + + # Verify Hook 3 output is grouped + sed -n "/Hook 3 Start/,/Hook 3 End/p" err.parallel >hook3_out && + test_line_count = 2 hook3_out +' + +test_expect_success 'git hook run -j1 runs hooks in series' ' + test_when_finished "rm -rf .git/hooks" && + + test_config hook.series-1.event "test-hook" && + test_config hook.series-1.command "echo 1" --add && + test_config hook.series-2.event "test-hook" && + test_config hook.series-2.command "echo 2" --add && + + mkdir -p .git/hooks && + write_script .git/hooks/test-hook <<-EOF && + echo 3 + EOF + + cat >expected <<-\EOF && + 1 + 2 + 3 + EOF + + git hook run --allow-unknown-hook-name -j1 test-hook 2>actual && + test_cmp expected actual +' + +test_expect_success 'git hook run -j2 runs hooks in parallel' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_when_finished "rm -rf .git/hooks" && + + mkdir -p .git/hooks && + write_sentinel_hook .git/hooks/test-hook && + + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + git hook run --allow-unknown-hook-name -j2 test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'git hook run -j2 overrides parallel=false' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + # hook-1 intentionally has no parallel=true + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + # hook-2 also has no parallel=true + + # -j2 overrides parallel=false; hooks run in parallel with a warning. + git hook run --allow-unknown-hook-name -j2 test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'git hook run -j2 warns for hooks not marked parallel=true' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command "true" && + # neither hook has parallel=true + + git hook run --allow-unknown-hook-name -j2 test-hook >out 2>err && + grep "hook .hook-1. is not marked as parallel=true" err && + grep "hook .hook-2. is not marked as parallel=true" err +' + +test_expect_success 'hook.jobs=1 config runs hooks in series' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + + # Use two configured hooks so the execution order is deterministic: + # hook-1 (sentinel) is listed before hook-2 (detector), so hook-1 + # always runs first even in serial mode. + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + + test_config hook.jobs 1 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook.jobs=2 config runs hooks in parallel' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_when_finished "rm -rf .git/hooks" && + + mkdir -p .git/hooks && + write_sentinel_hook .git/hooks/test-hook && + + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..parallel=true enables parallel execution' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..parallel=false (default) forces serial execution' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'one non-parallel hook forces the whole event to run serially' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + # hook-2 has no parallel=true: should force serial for all + + test_config hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'client hooks: pre-push parallel execution merges stdout to stderr' ' + test_when_finished "rm -rf remote-par stdout.actual stderr.actual" && + git init --bare remote-par && + git remote add origin-par remote-par && + test_commit par-commit && + mkdir -p .git/hooks && + setup_hooks pre-push && + test_config hook.jobs 2 && + git push origin-par HEAD:main >stdout.actual 2>stderr.actual && + check_stdout_merged_to_stderr pre-push +' + +test_expect_success 'client hooks: pre-push runs in parallel when hook.jobs > 1' ' + test_when_finished "rm -rf repo-parallel remote-parallel" && + git init --bare remote-parallel && + git init repo-parallel && + git -C repo-parallel remote add origin ../remote-parallel && + test_commit -C repo-parallel A && + + write_sentinel_hook repo-parallel/.git/hooks/pre-push && + git -C repo-parallel config hook.hook-2.event pre-push && + git -C repo-parallel config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + git -C repo-parallel config hook.hook-2.parallel true && + + git -C repo-parallel config hook.jobs 2 && + + git -C repo-parallel push origin HEAD >out 2>err && + echo parallel >expect && + test_cmp expect repo-parallel/hook.order +' + +test_expect_success 'hook.jobs=2 is ignored for force-serial hooks (pre-commit)' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event pre-commit && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event pre-commit && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + test_config hook.jobs 2 && + git commit --allow-empty -m "test: verify force-serial on pre-commit" && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs overrides hook.jobs for that event' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + # Global hook.jobs=1 (serial), but per-event override allows parallel. + test_config hook.jobs 1 && + test_config hook.test-hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo parallel >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs=1 forces serial even when hook.jobs>1' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + test_config hook.hook-1.parallel true && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + test_config hook.hook-2.parallel true && + + # Global hook.jobs=4 allows parallel, but per-event override forces serial. + test_config hook.jobs 4 && + test_config hook.test-hook.jobs 1 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs still requires hook..parallel=true' ' + test_when_finished "rm -f sentinel.started sentinel.done hook.order" && + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command \ + "touch sentinel.started; sleep 2; touch sentinel.done" && + # hook-1 intentionally has no parallel=true + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command \ + "$(sentinel_detector sentinel hook.order)" && + # hook-2 also has no parallel=true + + # Per-event jobs=2 but no hook has parallel=true: must still run serially. + test_config hook.test-hook.jobs 2 && + + git hook run --allow-unknown-hook-name test-hook >out 2>err && + echo serial >expect && + test_cmp expect hook.order +' + +test_expect_success 'hook..jobs warns when name has .command' ' + test_config hook.my-hook.command "true" && + test_config hook.my-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.my-hook.jobs.*friendly-name" err +' + +test_expect_success 'hook..jobs warns when name has .event' ' + test_config hook.my-hook.event test-hook && + test_config hook.my-hook.command "true" && + test_config hook.my-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.my-hook.jobs.*friendly-name" err +' + +test_expect_success 'hook..jobs warns when name has .parallel' ' + test_config hook.my-hook.event test-hook && + test_config hook.my-hook.command "true" && + test_config hook.my-hook.parallel true && + test_config hook.my-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.my-hook.jobs.*friendly-name" err +' + +test_expect_success 'hook..jobs does not warn for a real event name' ' + test_config hook.test-hook.jobs 2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep ! "friendly-name" err +' + +test_expect_success 'hook.jobs=-1 resolves to online_cpus()' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-1.parallel true && + + test_config hook.jobs -1 && + + cpus=$(test-tool online-cpus) && + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git hook run --allow-unknown-hook-name test-hook >out 2>err && + grep "\"region_enter\".*\"hook\".*\"test-hook\".*\"max:$cpus\"" trace.txt +' + +test_expect_success 'hook..jobs=-1 resolves to online_cpus()' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-1.parallel true && + + test_config hook.test-hook.jobs -1 && + + cpus=$(test-tool online-cpus) && + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git hook run --allow-unknown-hook-name test-hook >out 2>err && + grep "\"region_enter\".*\"hook\".*\"test-hook\".*\"max:$cpus\"" trace.txt +' + +test_expect_success 'git hook run -j-1 resolves to online_cpus()' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "true" && + test_config hook.hook-1.parallel true && + + cpus=$(test-tool online-cpus) && + GIT_TRACE2_EVENT="$(pwd)/trace.txt" \ + git hook run --allow-unknown-hook-name -j-1 test-hook >out 2>err && + grep "\"region_enter\".*\"hook\".*\"test-hook\".*\"max:$cpus\"" trace.txt +' + +test_expect_success 'hook.jobs rejects values less than -1' ' + test_config hook.jobs -2 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.jobs must be a positive integer or -1" err +' + +test_expect_success 'hook..jobs rejects values less than -1' ' + test_config hook.test-hook.jobs -5 && + git hook run --allow-unknown-hook-name --ignore-missing test-hook >out 2>err && + test_grep "hook.test-hook.jobs must be a positive integer or -1" err +' + +test_expect_success 'hook..enabled=false skips all hooks for event' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_must_be_empty out +' + +test_expect_success 'hook..enabled=true does not suppress hooks' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled true && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep "ran" err +' + +test_expect_success 'hook..enabled=false does not affect other events' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.other-event.enabled false && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep "ran" err +' + +test_expect_success 'hook..enabled=false still disables that hook' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo hook-1" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command "echo hook-2" && + test_config hook.hook-1.enabled false && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep ! "hook-1" err && + test_grep "hook-2" err +' + +test_expect_success 'git hook list shows event-disabled hooks as event-disabled' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.hook-2.event test-hook && + test_config hook.hook-2.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook list --allow-unknown-hook-name test-hook >actual && + test_grep "^event-disabled hook-1$" actual && + test_grep "^event-disabled hook-2$" actual +' + +test_expect_success 'git hook list shows scope with event-disabled' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook list --allow-unknown-hook-name --show-scope test-hook >actual && + test_grep "^local event-disabled hook-1$" actual +' + +test_expect_success 'git hook list still shows hooks when event is disabled' ' + test_config hook.hook-1.event test-hook && + test_config hook.hook-1.command "echo ran" && + test_config hook.test-hook.enabled false && + git hook list --allow-unknown-hook-name test-hook >actual && + test_grep "event-disabled" actual +' + +test_expect_success 'friendly-name matching known event name is rejected' ' + test_config hook.pre-commit.event pre-commit && + test_config hook.pre-commit.command "echo oops" && + test_must_fail git hook run pre-commit 2>err && + test_grep "collides with a known event name" err +' + +test_expect_success 'friendly-name matching known event name is rejected even for different event' ' + test_config hook.pre-commit.event post-commit && + test_config hook.pre-commit.command "echo oops" && + test_must_fail git hook run post-commit 2>err && + test_grep "collides with a known event name" err +' + +test_expect_success 'friendly-name matching unknown event warns' ' + test_config hook.test-hook.event test-hook && + test_config hook.test-hook.command "echo ran" && + git hook run --allow-unknown-hook-name test-hook >out 2>err && + test_grep "same as its event" err +' + +test_expect_success 'hooks in parallel that do not read input' ' + # Add this to our $PATH to avoid having to write the whole trash + # directory into our config options, which would require quoting. + mkdir bin && + PATH=$PWD/bin:$PATH && + + write_script bin/hook-fast <<-\EOF && + # This hook does not read its input, so the parent process + # may see SIGPIPE if it is not ignored. It should happen + # relatively quickly. + exit 0 + EOF + + write_script bin/hook-slow <<-\EOF && + # This hook is slow, so we expect it to still be running + # when the other hook has exited (and the parent has a pipe error + # writing to it). + # + # So we want to be slow enough that we expect this to happen, but not + # so slow that the test takes forever. 1 second is probably enough + # in practice (and if it is occasionally not on a loaded system, we + # will err on the side of having the test pass). + sleep 1 + exit 0 + EOF + + git init --bare parallel.git && + git -C parallel.git config hook.fast.command "hook-fast" && + git -C parallel.git config hook.fast.event pre-receive && + git -C parallel.git config hook.fast.parallel true && + git -C parallel.git config hook.slow.command "hook-slow" && + git -C parallel.git config hook.slow.event pre-receive && + git -C parallel.git config hook.slow.parallel true && + git -C parallel.git config hook.jobs 2 && + + git push ./parallel.git "+refs/heads/*:refs/heads/*" +' + test_done diff --git a/t/t5409-colorize-remote-messages.sh b/t/t5409-colorize-remote-messages.sh index fa5de4500a4f50..3010913bb113e4 100755 --- a/t/t5409-colorize-remote-messages.sh +++ b/t/t5409-colorize-remote-messages.sh @@ -98,4 +98,96 @@ test_expect_success 'fallback to color.ui' ' grep "error: error" decoded ' +test_expect_success 'disallow (color) control sequences in sideband' ' + write_script .git/color-me-surprised <<-\EOF && + printf "error: Have you \\033[31mread\\033[m this?\\a\\n" >&2 + exec "$@" + EOF + test_config_global uploadPack.packObjectsHook ./color-me-surprised && + test_commit need-at-least-one-commit && + + git clone --no-local . throw-away 2>stderr && + test_decode_color decoded && + test_grep RED decoded && + test_grep "\\^G" stderr && + tr -dc "\\007" actual && + test_must_be_empty actual && + + rm -rf throw-away && + git -c sideband.allowControlCharacters=false \ + clone --no-local . throw-away 2>stderr && + test_decode_color decoded && + test_grep ! RED decoded && + test_grep "\\^G" stderr && + + rm -rf throw-away && + git -c sideband.allowControlCharacters clone --no-local . throw-away 2>stderr && + test_decode_color decoded && + test_grep RED decoded && + tr -dc "\\007" actual && + test_file_not_empty actual +' + +test_decode_csi() { + awk '{ + while (match($0, /\033/) != 0) { + printf "%sCSI ", substr($0, 1, RSTART-1); + $0 = substr($0, RSTART + RLENGTH, length($0) - RSTART - RLENGTH + 1); + } + print + }' +} + +test_expect_success 'control sequences in sideband allowed by default' ' + write_script .git/color-me-surprised <<-\EOF && + printf "error: \\033[31mcolor\\033[m\\033[Goverwrite\\033[Gerase\\033[K\\033?25l\\n" >&2 + exec "$@" + EOF + test_config_global uploadPack.packObjectsHook ./color-me-surprised && + test_commit need-at-least-one-commit-at-least && + + rm -rf throw-away && + git clone --no-local . throw-away 2>stderr && + test_decode_color color-decoded && + test_decode_csi decoded && + test_grep ! "CSI \\[K" decoded && + test_grep ! "CSI \\[G" decoded && + test_grep "\\^\\[?25l" decoded && + + rm -rf throw-away && + git -c sideband.allowControlCharacters=erase,cursor,color \ + clone --no-local . throw-away 2>stderr && + test_decode_color color-decoded && + test_decode_csi decoded && + test_grep "RED" decoded && + test_grep "CSI \\[K" decoded && + test_grep "CSI \\[G" decoded && + test_grep ! "\\^\\[\\[K" decoded && + test_grep ! "\\^\\[\\[G" decoded +' + +test_expect_success 'allow all control sequences for a specific URL' ' + write_script .git/eraser <<-\EOF && + printf "error: Ohai!\\r\\033[K" >&2 + exec "$@" + EOF + test_config_global uploadPack.packObjectsHook ./eraser && + test_commit one-more-please && + + rm -rf throw-away && + git clone --no-local . throw-away 2>stderr && + test_decode_color color-decoded && + test_decode_csi decoded && + test_grep ! "CSI \\[K" decoded && + test_grep "\\^\\[\\[K" decoded && + + rm -rf throw-away && + git -c "sideband.file://.allowControlCharacters=true" \ + clone --no-local "file://$PWD" throw-away 2>stderr && + test_decode_color color-decoded && + test_decode_csi decoded && + test_grep "CSI \\[K" decoded && + test_grep ! "\\^\\[\\[K" decoded +' + test_done diff --git a/t/t5710-promisor-remote-capability.sh b/t/t5710-promisor-remote-capability.sh index 357822c01a7530..b404ad9f0a9e3d 100755 --- a/t/t5710-promisor-remote-capability.sh +++ b/t/t5710-promisor-remote-capability.sh @@ -76,6 +76,31 @@ copy_to_lop () { cp "$path" "$path2" } +# On Windows, `pwd` returns a path like 'D:/foo/bar'. Prepend '/' to turn +# it into '/D:/foo/bar', which is what git expects in file:// URLs on Windows. +# On Unix, the path already starts with '/', so this is a no-op. +pwd_path=$(pwd) +case "$pwd_path" in +[a-zA-Z]:*) pwd_path="/$pwd_path" ;; +esac + +# Allowed characters: alphanumeric, standard path/URI (_ . ~ / : -), +# and those percent-encoded below (% space = , ;) +rest=$(printf "%s" "$pwd_path" | tr -d 'a-zA-Z0-9_.~/:% =,;-') +if test -n "$rest" +then + skip_all="PWD contains unsupported special characters" + test_done +fi + +TRASH_DIRECTORY_URL="file://$pwd_path" + +encoded_path=$(printf "%s" "$pwd_path" | + sed -e 's/%/%25/g' -e 's/ /%20/g' -e 's/=/%3D/g' \ + -e 's/;/%3B/g' -e 's/,/%2C/g') + +ENCODED_TRASH_DIRECTORY_URL="file://$encoded_path" + test_expect_success "setup for testing promisor remote advertisement" ' # Create another bare repo called "lop" (for Large Object Promisor) git init --bare lop && @@ -88,7 +113,7 @@ test_expect_success "setup for testing promisor remote advertisement" ' initialize_server 1 "$oid" && # Configure lop as promisor remote for server - git -C server remote add lop "file://$(pwd)/lop" && + git -C server remote add lop "$TRASH_DIRECTORY_URL/lop" && git -C server config remote.lop.promisor true && git -C lop config uploadpack.allowFilter true && @@ -104,7 +129,7 @@ test_expect_success "clone with promisor.advertise set to 'true'" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=All \ --no-local --filter="blob:limit=5k" server client && @@ -119,7 +144,7 @@ test_expect_success "clone with promisor.advertise set to 'false'" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=All \ --no-local --filter="blob:limit=5k" server client && @@ -137,7 +162,7 @@ test_expect_success "clone with promisor.acceptfromserver set to 'None'" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=None \ --no-local --filter="blob:limit=5k" server client && @@ -156,8 +181,8 @@ test_expect_success "init + fetch with promisor.advertise set to 'true'" ' git -C client init && git -C client config remote.lop.promisor true && git -C client config remote.lop.fetch "+refs/heads/*:refs/remotes/lop/*" && - git -C client config remote.lop.url "file://$(pwd)/lop" && - git -C client config remote.server.url "file://$(pwd)/server" && + git -C client config remote.lop.url "$TRASH_DIRECTORY_URL/lop" && + git -C client config remote.server.url "$TRASH_DIRECTORY_URL/server" && git -C client config remote.server.fetch "+refs/heads/*:refs/remotes/server/*" && git -C client config promisor.acceptfromserver All && GIT_NO_LAZY_FETCH=0 git -C client fetch --filter="blob:limit=5k" server && @@ -166,6 +191,75 @@ test_expect_success "init + fetch with promisor.advertise set to 'true'" ' check_missing_objects server 1 "$oid" ' +test_expect_success "clone with two promisors but only one advertised" ' + git -C server config promisor.advertise true && + test_when_finished "rm -rf client unused_lop" && + + # Create a promisor that will be configured but not be used + git init --bare unused_lop && + + # Clone from server to create a client + GIT_TRACE="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \ + -c remote.unused_lop.promisor=true \ + -c remote.unused_lop.fetch="+refs/heads/*:refs/remotes/unused_lop/*" \ + -c remote.unused_lop.url="$TRASH_DIRECTORY_URL/unused_lop" \ + -c remote.lop.promisor=true \ + -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ + -c promisor.acceptfromserver=All \ + --no-local --filter="blob:limit=5k" server client && + + # Check that "unused_lop" appears before "lop" in the config + printf "remote.%s.promisor true\n" "unused_lop" "lop" "origin" >expect && + git -C client config get --all --show-names --regexp "^remote\..*\.promisor$" >actual && + test_cmp expect actual && + + # Check that "lop" was tried + test_grep " fetch lop " trace && + # Check that "unused_lop" was not contacted + # This means "lop", the accepted promisor, was tried first + test_grep ! " fetch unused_lop " trace && + + # Check that the largest object is still missing on the server + check_missing_objects server 1 "$oid" +' + +test_expect_success "init + fetch two promisors but only one advertised" ' + git -C server config promisor.advertise true && + test_when_finished "rm -rf client unused_lop" && + + # Create a promisor that will be configured but not be used + git init --bare unused_lop && + + mkdir client && + git -C client init && + git -C client config remote.unused_lop.promisor true && + git -C client config remote.unused_lop.fetch "+refs/heads/*:refs/remotes/unused_lop/*" && + git -C client config remote.unused_lop.url "$TRASH_DIRECTORY_URL/unused_lop" && + git -C client config remote.lop.promisor true && + git -C client config remote.lop.fetch "+refs/heads/*:refs/remotes/lop/*" && + git -C client config remote.lop.url "$TRASH_DIRECTORY_URL/lop" && + git -C client config remote.server.url "$TRASH_DIRECTORY_URL/server" && + git -C client config remote.server.fetch "+refs/heads/*:refs/remotes/server/*" && + git -C client config promisor.acceptfromserver All && + + # Check that "unused_lop" appears before "lop" in the config + printf "remote.%s.promisor true\n" "unused_lop" "lop" >expect && + git -C client config get --all --show-names --regexp "^remote\..*\.promisor$" >actual && + test_cmp expect actual && + + GIT_TRACE="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git -C client fetch --filter="blob:limit=5k" server && + + # Check that "lop" was tried + test_grep " fetch lop " trace && + # Check that "unused_lop" was not contacted + # This means "lop", the accepted promisor, was tried first + test_grep ! " fetch unused_lop " trace && + + # Check that the largest object is still missing on the server + check_missing_objects server 1 "$oid" +' + test_expect_success "clone with promisor.acceptfromserver set to 'KnownName'" ' git -C server config promisor.advertise true && test_when_finished "rm -rf client" && @@ -173,7 +267,7 @@ test_expect_success "clone with promisor.acceptfromserver set to 'KnownName'" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=KnownName \ --no-local --filter="blob:limit=5k" server client && @@ -188,7 +282,7 @@ test_expect_success "clone with 'KnownName' and different remote names" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.serverTwo.promisor=true \ -c remote.serverTwo.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.serverTwo.url="file://$(pwd)/lop" \ + -c remote.serverTwo.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=KnownName \ --no-local --filter="blob:limit=5k" server client && @@ -225,7 +319,7 @@ test_expect_success "clone with promisor.acceptfromserver set to 'KnownUrl'" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=KnownUrl \ --no-local --filter="blob:limit=5k" server client && @@ -242,7 +336,7 @@ test_expect_success "clone with 'KnownUrl' and different remote urls" ' # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/serverTwo" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/serverTwo" \ -c promisor.acceptfromserver=KnownUrl \ --no-local --filter="blob:limit=5k" server client && @@ -257,7 +351,7 @@ test_expect_success "clone with 'KnownUrl' and url not configured on the server" git -C server config promisor.advertise true && test_when_finished "rm -rf client" && - test_when_finished "git -C server config set remote.lop.url \"file://$(pwd)/lop\"" && + test_when_finished "git -C server config set remote.lop.url \"$TRASH_DIRECTORY_URL/lop\"" && git -C server config unset remote.lop.url && # Clone from server to create a client @@ -266,7 +360,7 @@ test_expect_success "clone with 'KnownUrl' and url not configured on the server" # missing, so the remote name will be used instead which will fail. test_must_fail env GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=KnownUrl \ --no-local --filter="blob:limit=5k" server client && @@ -278,7 +372,7 @@ test_expect_success "clone with 'KnownUrl' and empty url, so not advertised" ' git -C server config promisor.advertise true && test_when_finished "rm -rf client" && - test_when_finished "git -C server config set remote.lop.url \"file://$(pwd)/lop\"" && + test_when_finished "git -C server config set remote.lop.url \"$TRASH_DIRECTORY_URL/lop\"" && git -C server config set remote.lop.url "" && # Clone from server to create a client @@ -287,7 +381,7 @@ test_expect_success "clone with 'KnownUrl' and empty url, so not advertised" ' # so the remote name will be used instead which will fail. test_must_fail env GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=KnownUrl \ --no-local --filter="blob:limit=5k" server client && @@ -311,13 +405,12 @@ test_expect_success "clone with promisor.sendFields" ' GIT_TRACE_PACKET="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \ -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=All \ --no-local --filter="blob:limit=5k" server client && # Check that fields are properly transmitted - ENCODED_URL=$(echo "file://$(pwd)/lop" | sed -e "s/ /%20/g") && - PR1="name=lop,url=$ENCODED_URL,partialCloneFilter=blob:none" && + PR1="name=lop,url=$ENCODED_TRASH_DIRECTORY_URL/lop,partialCloneFilter=blob:none" && PR2="name=otherLop,url=https://invalid.invalid,partialCloneFilter=blob:limit=10k,token=fooBar" && test_grep "clone< promisor-remote=$PR1;$PR2" trace && test_grep "clone> promisor-remote=lop;otherLop" trace && @@ -342,15 +435,14 @@ test_expect_success "clone with promisor.checkFields" ' GIT_TRACE_PACKET="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \ -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c remote.lop.partialCloneFilter="blob:none" \ -c promisor.acceptfromserver=All \ -c promisor.checkFields=partialcloneFilter \ --no-local --filter="blob:limit=5k" server client && # Check that fields are properly transmitted - ENCODED_URL=$(echo "file://$(pwd)/lop" | sed -e "s/ /%20/g") && - PR1="name=lop,url=$ENCODED_URL,partialCloneFilter=blob:none" && + PR1="name=lop,url=$ENCODED_TRASH_DIRECTORY_URL/lop,partialCloneFilter=blob:none" && PR2="name=otherLop,url=https://invalid.invalid,partialCloneFilter=blob:limit=10k,token=fooBar" && test_grep "clone< promisor-remote=$PR1;$PR2" trace && test_grep "clone> promisor-remote=lop" trace && @@ -380,7 +472,7 @@ test_expect_success "clone with promisor.storeFields=partialCloneFilter" ' GIT_TRACE_PACKET="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \ -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c remote.lop.token="fooYYY" \ -c remote.lop.partialCloneFilter="blob:none" \ -c promisor.acceptfromserver=All \ @@ -432,7 +524,7 @@ test_expect_success "clone and fetch with --filter=auto" ' GIT_TRACE_PACKET="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \ -c remote.lop.promisor=true \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=All \ --no-local --filter=auto server client 2>err && @@ -489,7 +581,7 @@ test_expect_success "clone with promisor.advertise set to 'true' but don't delet # Clone from server to create a client GIT_NO_LAZY_FETCH=0 git clone -c remote.lop.promisor=true \ -c remote.lop.fetch="+refs/heads/*:refs/remotes/lop/*" \ - -c remote.lop.url="file://$(pwd)/lop" \ + -c remote.lop.url="$TRASH_DIRECTORY_URL/lop" \ -c promisor.acceptfromserver=All \ --no-local --filter="blob:limit=5k" server client && diff --git a/transport.c b/transport.c index e53936d87b641f..9cde4a4e43a391 100644 --- a/transport.c +++ b/transport.c @@ -29,6 +29,7 @@ #include "object-name.h" #include "color.h" #include "bundle-uri.h" +#include "sideband.h" static enum git_colorbool transport_use_color = GIT_COLOR_UNKNOWN; static char transport_colors[][COLOR_MAXLEN] = { @@ -1246,6 +1247,8 @@ struct transport *transport_get(struct remote *remote, const char *url) ret->hash_algo = &hash_algos[GIT_HASH_SHA1_LEGACY]; + sideband_apply_url_config(ret->url); + return ret; } @@ -1391,8 +1394,10 @@ static int run_pre_push_hook(struct transport *transport, opt.feed_pipe_cb_data_free = pre_push_hook_data_free; /* - * pre-push hooks expect stdout & stderr to be separate, so don't merge - * them to keep backwards compatibility with existing hooks. + * pre-push hooks keep stdout and stderr separate by default for + * backwards compatibility. When the user opts into parallel execution + * via hook.jobs > 1 or -j, get_hook_jobs() will set stdout_to_stderr=1 + * automatically so run-command can de-interleave the outputs. */ opt.stdout_to_stderr = 0;