+B<mr> [options] [online|offline]
+
+B<mr> [options] remember action [params ...]
+
+=head1 DESCRIPTION
+
+B<mr> is a tool to manage all your version control repos. It can checkout,
+update, or perform other actions on a set of repositories as if they were
+one combined repository. It supports any combination of subversion, git,
+cvs, mercurial, bzr, darcs, fossil and veracity repositories, and support
+for other version control systems can easily be added.
+
+B<mr> cds into and operates on all registered repositories at or below your
+working directory. Or, if you are in a subdirectory of a repository that
+contains no other registered repositories, it will stay in that directory,
+and work on only that repository,
+
+B<mr> is configured by .mrconfig files, which list the repositories. It
+starts by reading the .mrconfig file in your home directory, and this can
+in turn chain load .mrconfig files from repositories. It also automatically
+looks for a .mrconfig file in the current directory, or in one of its
+parent directories.
+
+These predefined commands should be fairly familiar to users of any version
+control system:
+
+=over 4
+
+=item checkout (or co)
+
+Checks out any repositories that are not already checked out.
+
+=item update
+
+Updates each repository from its configured remote repository.
+
+If a repository isn't checked out yet, it will first check it out.
+
+=item status
+
+Displays a status report for each repository, showing what
+uncommitted changes are present in the repository. For distributed version
+control systems, also shows unpushed local branches.
+
+=item commit (or ci)
+
+Commits changes to each repository. (By default, changes are pushed to the
+remote repository too, when using distributed systems like git. If you
+don't like this default, you can change it in your .mrconfig, or use record
+instead.)
+
+The optional -m parameter allows specifying a commit message.
+
+=item record
+
+Records changes to the local repository, but does not push them to the
+remote repository. Only supported for distributed version control systems.
+
+The optional -m parameter allows specifying a commit message.
+
+=item fetch
+
+Fetches from each repository's remote repository, but does not
+update the working copy. Only supported for some distributed version
+control systems.
+
+=item push
+
+Pushes committed local changes to the remote repository. A no-op for
+centralized version control systems.
+
+=item diff
+
+Show a diff of uncommitted changes.
+
+=item log
+
+Show the commit log.
+
+=item grep pattern
+
+Searches for a pattern in each repository using the grep subcommand. Uses
+ack-grep on VCS that do not have their own.
+
+=item run command [param ...]
+
+Runs the specified command in each repository.
+
+=back
+
+These commands are also available:
+
+=over 4
+
+=item bootstrap src [directory]
+
+Causes mr to retrieve the source C<src> and use it as a .mrconfig file to
+checkout the repositories listed in it, into the specified directory.
+
+B<mr> understands several types of sources:
+
+=over 4
+
+=item URL for curl
+
+C<src> may be an URL understood by B<curl>.
+
+=item copy via ssh
+
+To use B<scp> to download, the C<src> may have the form
+C<ssh://[user@]host:file>.
+
+=item local file
+
+You can retrieve the config file by other means and pass its B<path> as C<src>.
+
+=item standard input
+
+If source C<src> consists in a single dash C<->, config file is read from
+standard input.
+
+=back
+
+The directory will be created if it does not exist. If no directory is
+specified, the current directory will be used.
+
+As a special case, if source C<src> includes a repository named ".", that
+is checked out into the top of the specified directory.
+
+=item list (or ls)
+
+List the repositories that mr will act on.
+
+=item register
+
+Register an existing repository in a mrconfig file. By default, the
+repository in the current directory is registered, or you can specify a
+directory to register.
+
+The mrconfig file that is modified is chosen by either the -c option, or by
+looking for the closest known one at or in a parent of the current directory.
+
+=item config
+
+Adds, modifies, removes, or prints a value from a mrconfig file. The next
+parameter is the name of the section the value is in. To add or modify
+values, use one or more instances of "parameter=value". Use "parameter=" to
+remove a parameter. Use just "parameter" to get the value of a parameter.
+
+For example, to add (or edit) a repository in src/foo:
+
+ mr config src/foo checkout="svn co svn://example.com/foo/trunk foo"
+
+To show the command that mr uses to update the repository in src/foo:
+
+ mr config src/foo update
+
+To see the built-in library of shell functions contained in mr:
+
+ mr config DEFAULT lib
+
+The mrconfig file that is used is chosen by either the -c option, or by
+looking for the closest known one at or in a parent of the current directory.
+
+=item offline
+
+Advises mr that it is in offline mode. Any commands that fail in
+offline mode will be remembered, and retried when mr is told it's online.
+
+=item online
+
+Advices mr that it is in online mode again. Commands that failed while in
+offline mode will be re-run.
+
+=item remember
+
+Remember a command, to be run later when mr re-enters online mode. This
+implicitly puts mr into offline mode. The command can be any regular mr
+command. This is useful when you know that a command will fail due to being
+offline, and so don't want to run it right now at all, but just remember
+to run it when you go back online.
+
+=item help
+
+Displays this help.
+
+=back
+
+Actions can be abbreviated to any unambiguous substring, so
+"mr st" is equivalent to "mr status", and "mr up" is equivalent to "mr
+update"
+
+Additional parameters can be passed to most commands, and are passed on
+unchanged to the underlying version control system. This is mostly useful
+if the repositories mr will act on all use the same version control
+system.
+
+=head1 OPTIONS
+
+=over 4
+
+=item -d directory
+
+=item --directory directory
+
+Specifies the topmost directory that B<mr> should work in. The default is
+the current working directory.
+
+=item -c mrconfig
+
+=item --config mrconfig
+
+Use the specified mrconfig file. The default is to use both F<~/.mrconfig>
+as well as look for a F<.mrconfig> file in the current directory, or in one
+of its parent directories.
+
+=item -f
+
+=item --force
+
+Force mr to act on repositories that would normally be skipped due to their
+configuration.
+
+=item -v
+
+=item --verbose
+
+Be verbose.
+
+=item -m
+
+=item --minimal
+
+Minimise output. If a command fails or there is any output then the usual
+output will be shown.
+
+=item -q
+
+=item --quiet
+
+Be quiet. This suppresses mr's usual output, as well as any output from
+commands that are run (including stderr output). If a command fails,
+the output will be shown.
+
+=item -k
+
+=item --insecure
+
+Accept untrusted SSL certificates when bootstrapping.
+
+=item -s
+
+=item --stats
+
+Expand the statistics line displayed at the end to include information
+about exactly which repositories failed and were skipped, if any.
+
+=item -i
+
+=item --interactive
+
+Interactive mode. If a repository fails to be processed, a subshell will be
+started which you can use to resolve or investigate the problem. Exit the
+subshell to continue the mr run.
+
+=item -n [number]
+
+=item --no-recurse [number]
+
+If no number if specified, just operate on the repository for the current
+directory, do not recurse into deeper repositories.
+
+If a number is specified, will recurse into repositories at most that many
+subdirectories deep. For example, with -n 2 it would recurse into ./src/foo,
+but not ./src/packages/bar.
+
+=item -j [number]
+
+=item --jobs [number]
+
+Run the specified number of jobs in parallel, or an unlimited number of jobs
+with no number specified. This can greatly speed up operations such as updates.
+It is not recommended for interactive operations.
+
+Note that running more than 10 jobs at a time is likely to run afoul of
+ssh connection limits. Running between 3 and 5 jobs at a time will yield
+a good speedup in updates without loading the machine too much.
+
+=item -t
+
+=item --trust-all
+
+Trust all mrconfig files even if they are not listed in F<~/.mrtrust>.
+Use with caution.
+
+=item -p
+
+=item --path
+
+This obsolete flag is ignored.
+
+=back
+
+=head1 MRCONFIG FILES
+
+Here is an example F<.mrconfig> file:
+
+ [src]
+ checkout = svn checkout svn://svn.example.com/src/trunk src
+ chain = true
+
+ [src/linux-2.6]
+ checkout = git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git &&
+ cd linux-2.6 &&
+ git checkout -b mybranch origin/master
+
+The F<.mrconfig> file uses a variant of the INI file format. Lines
+starting with "#" are comments. Values can be continued to the
+following line by indenting the line with whitespace.
+
+The C<DEFAULT> section allows setting default values for the sections that
+come after it.
+
+The C<ALIAS> section allows adding aliases for actions. Each parameter
+is an alias, and its value is the action to use.
+
+All other sections add repositories. The section header specifies the
+directory where the repository is located. This is relative to the directory
+that contains the mrconfig file, but you can also choose to use absolute
+paths. (Note that you can use environment variables in section names; they
+will be passed through the shell for expansion. For example,
+C<[$HOSTNAME]>, or C<[${HOSTNAME}foo]>).
+
+Within a section, each parameter defines a shell command to run to handle a
+given action. mr contains default handlers for "update", "status",
+"commit", and other standard actions.
+
+Normally you only need to specify what to do for "checkout". Here you
+specify the command to run in order to create a checkout of the repository.
+The command will be run in the parent directory, and must create the
+repository's directory. So use C<git clone>, C<svn checkout>, C<bzr branch>
+or C<bzr checkout> (for a bound branch), etc.
+
+Note that these shell commands are run in a C<set -e> shell
+environment, where any additional parameters you pass are available in
+C<$@>. All commands other than "checkout" are run inside the repository,
+though not necessarily at the top of it.
+
+The C<MR_REPO> environment variable is set to the path to the top of the
+repository. (For the "register" action, "MR_REPO" is instead set to the
+basename of the directory that should be created when checking the
+repository out.)
+
+The C<MR_CONFIG> environment variable is set to the .mrconfig file
+that defines the repo being acted on, or, if the repo is not yet in a config
+file, the F<.mrconfig> file that should be modified to register the repo.
+
+The C<MR_ACTION> environment variable is set to the command being run
+(update, checkout, etc).
+
+A few parameters have special meanings:
+
+=over 4
+
+=item skip
+
+If the "skip" parameter is set and its command returns true, then B<mr>
+will skip acting on that repository. The command is passed the action
+name in C<$1>.
+
+Here are two examples. The first skips the repo unless
+mr is run by joey. The second uses the hours_since function
+(included in mr's built-in library) to skip updating the repo unless it's
+been at least 12 hours since the last update.
+
+ [mystuff]
+ checkout = ...
+ skip = test `whoami` != joey
+
+ [linux]
+ checkout = ...
+ skip = [ "$1" = update ] && ! hours_since "$1" 12
+
+Another way to use skip is for a lazy checkout. This makes mr skip
+operating on a repo unless it already exists. To enable the
+repo, you have to explicitly check it out (using "mr --force -d foo checkout").
+
+ [foo]
+ checkout = ...
+ skip = lazy
+
+=item order
+
+The "order" parameter can be used to override the default ordering of
+repositories. The default order value is 10. Use smaller values to make
+repositories be processed earlier, and larger values to make repositories
+be processed later.
+
+Note that if a repository is located in a subdirectory of another
+repository, ordering it to be processed earlier is not recommended.
+
+=item chain
+
+If the "chain" parameter is set and its command returns true, then B<mr>
+will try to load a F<.mrconfig> file from the root of the repository.
+
+=item include
+
+If the "include" parameter is set, its command is ran, and should output
+additional mrconfig file content. The content is included as if it were
+part of the including file.
+
+Unlike all other parameters, this parameter does not need to be placed
+within a section.
+
+B<mr> ships several libraries that can be included to add support for
+additional version control type things (unison, git-svn, git-fake-bare,
+git-subtree). To include them all, you could use:
+
+ include = cat /usr/share/mr/*
+
+See the individual files for details.
+
+=item deleted
+
+If the "deleted" parameter is set and its command returns true, then
+B<mr> will treat the repository as deleted. It won't ever actually delete
+the repository, but it will warn if it sees the repository's directory.
+This is useful when one mrconfig file is shared among multiple machines,
+to keep track of and remember to delete old repositories.
+
+=item lib
+
+The "lib" parameter can specify some shell code that will be run
+before each command, this can be a useful way to define shell
+functions for other commands to use.
+
+Unlike most other parameters, this can be specified multiple times, in
+which case the chunks of shell code are accumulatively concatenated
+together.
+
+=item fixups
+
+If the "fixups" parameter is set, its command is run whenever a repository
+is checked out, or updated. This provides an easy way to do things
+like permissions fixups, or other tweaks to the repository content,
+whenever the repository is changed.
+
+=item VCS_action
+
+When looking for a command to run for a given action, mr first looks for
+a parameter with the same name as the action. If that is not found, it
+looks for a parameter named "VCS_action" (substituting in the name of the
+version control system and the action).
+
+Internally, mr has settings for "git_update", "svn_update", etc. To change
+the action that is performed for a given version control system, you can
+override these VCS specific actions. To add a new version control system,
+you can just add VCS specific actions for it.
+
+=item pre_ and post_
+
+If a "pre_action" parameter is set, its command is run before mr performs the
+specified action. Similarly, "post_action" parameters are run after mr
+successfully performs the specified action. For example, "pre_commit" is
+run before committing; "post_update" is run after updating.
+
+=item _append
+
+Any parameter can be suffixed with C<_append>, to add an additional value
+to the existing value of the parameter. In this way, actions
+can be constructed accumulatively.
+
+=item VCS_test
+
+The name of the version control system is itself determined by
+running each defined "VCS_test" action, until one succeeds.
+
+=back
+
+=head1 UNTRUSTED MRCONFIG FILES
+
+Since mrconfig files can contain arbitrary shell commands, they can do
+anything. This flexibility is good, but it also allows a malicious mrconfig
+file to delete your whole home directory. Such a file might be contained
+inside a repository that your main F<~/.mrconfig> checks out. To
+avoid worries about evil commands in a mrconfig file, mr defaults to
+reading all mrconfig files other than the main F<~/.mrconfig> in untrusted
+mode. In untrusted mode, mrconfig files are limited to running only known
+safe commands (like "git clone") in a carefully checked manner.
+
+To configure mr to trust other mrconfig files, list them in F<~/.mrtrust>.
+One mrconfig file should be listed per line. Either the full pathname
+should be listed, or the pathname can start with F<~/> to specify a file
+relative to your home directory.
+
+=head1 OFFLINE LOG FILE
+
+The F<~/.mrlog> file contains commands that mr has remembered to run later,
+due to being offline. You can delete or edit this file to remove commands,
+or even to add other commands for 'mr online' to run. If the file is
+present, mr assumes it is in offline mode.
+
+=head1 EXTENSIONS
+
+mr can be extended to support things such as unison and git-svn. Some
+files providing such extensions are available in F</usr/share/mr/>. See
+the documentation in the files for details about using them.
+
+=head1 EXIT STATUS
+
+mr returns nonzero if a command failed in any of the repositories.
+
+=head1 AUTHOR
+
+Copyright 2007-2011 Joey Hess <joey@kitenet.net>
+
+Licensed under the GNU GPL version 2 or higher.
+
+http://myrepos.branchable.com/
+
+=cut
+
+use warnings;
+use strict;
+use Getopt::Long;
+use Cwd qw(getcwd abs_path);
+use File::Basename;
+
+# things that can happen when mr runs a command
+use constant {
+ OK => 0,
+ FAILED => 1,
+ SKIPPED => 2,
+ ABORT => 3,
+};
+
+# configurables
+my $config_overridden=0;
+my $verbose=0;
+my $minimal=0;
+my $quiet=0;
+my $stats=0;
+my $force=0;
+my $insecure=0;
+my $interactive=0;
+my $max_depth;
+my $no_chdir=0;
+my $jobs=1;
+my $trust_all=0;
+my $directory=getcwd();
+my $terminal=-t STDOUT && eval{require IO::Pty::Easy;IO::Pty::Easy->import();1;};
+
+my $HOME_MR_CONFIG = "$ENV{HOME}/.mrconfig";
+$ENV{MR_CONFIG}=find_mrconfig();
+
+# globals :-(
+my %config;
+my %configfiles;
+my %knownactions;
+my %alias;
+my (@ok, @failed, @skipped);
+
+main();
+
+sub shellquote {
+ my $i=shift;
+ $i=~s/'/'"'"'/g;
+ return "'$i'";
+}
+
+# Runs a shell command using a supplied function.
+# The lib will be included in the shell command line, and any params
+# will be available in the shell as $1, $2, etc.
+my $lastlib;
+sub runsh {
+ my ($action, $topdir, $subdir, $command, $params, $runner) = @_;
+
+ # optimisation: avoid running the shell for true and false
+ if ($command =~ /^\s*true\s*$/) {
+ $?=0;
+ return 0;
+ }
+ elsif ($command =~ /^\s*false\s*$/) {
+ $?=0;
+ return 1;
+ }
+
+ my $quotedparams=join(" ", (map { shellquote($_) } @$params));
+ my $lib=exists $config{$topdir}{$subdir}{lib} ?
+ $config{$topdir}{$subdir}{lib}."\n" : "";
+ if ($verbose && (! defined $lastlib || $lastlib ne $lib)) {
+ print "mr library now: >>$lib<<\n";
+ $lastlib=$lib;
+ }
+ my $shellcode="set -e;".$lib.
+ "my_sh(){ $command\n }; my_sh $quotedparams";
+ print "mr $action: running $action >>$command<<\n" if $verbose;
+ $runner->($shellcode);
+}
+
+my %perl_cache;
+sub perl {
+ my $id=shift;
+ my $s=shift;
+ if ($s =~ m/^perl:\s+(.*)/s) {
+ return $perl_cache{$1} if exists $perl_cache{$1};
+ my $sub=eval "sub {$1}";
+ if (! defined $sub) {
+ print STDERR "mr: bad perl code in $id: $@\n";
+ }
+ return $perl_cache{$1} = $sub;
+ }
+ return undef;
+}
+
+my %vcs;
+sub vcs_test {
+ my ($action, $dir, $topdir, $subdir) = @_;
+
+ if (exists $vcs{$dir}) {
+ return $vcs{$dir};
+ }
+
+ my $test="";
+ my %perltest;
+ foreach my $vcs_test (
+ sort {
+ length $a <=> length $b
+ ||
+ $a cmp $b
+ } grep { /_test$/ } keys %{$config{$topdir}{$subdir}}) {
+ my ($vcs)=$vcs_test =~ /(.*)_test/;
+ my $p=perl($vcs_test, $config{$topdir}{$subdir}{$vcs_test});
+ if (defined $p) {
+ $perltest{$vcs}=$p;
+ }
+ else {
+ $test="my_$vcs_test() {\n$config{$topdir}{$subdir}{$vcs_test}\n}\n".$test;
+ $test.="if my_$vcs_test; then echo $vcs; fi\n";
+ }
+ }
+
+ my @vcs;
+ foreach my $vcs (keys %perltest) {
+ if ($perltest{$vcs}->()) {
+ push @vcs, $vcs;
+ }
+ }
+
+ push @vcs, split(/\n/,
+ runsh("vcs test", $topdir, $subdir, $test, [], sub {
+ my $sh=shift;
+ my $ret=`$sh`;
+ return $ret;
+ })) if length $test;
+ if (@vcs > 1) {
+ print STDERR "mr $action: found multiple possible repository types (@vcs) for ".fulldir($topdir, $subdir)."\n";
+ return undef;
+ }
+ if (! @vcs) {
+ return $vcs{$dir}=undef;
+ }
+ else {
+ return $vcs{$dir}=$vcs[0];
+ }
+}
+
+sub findcommand {
+ my ($action, $dir, $topdir, $subdir, $is_checkout) = @_;
+
+ if (exists $config{$topdir}{$subdir}{$action}) {
+ return $config{$topdir}{$subdir}{$action};
+ }
+
+ if ($is_checkout) {
+ return undef;
+ }
+
+ my $vcs=vcs_test(@_);
+
+ if (defined $vcs &&
+ exists $config{$topdir}{$subdir}{$vcs."_".$action}) {
+ return $config{$topdir}{$subdir}{$vcs."_".$action};
+ }
+ else {
+ return undef;
+ }
+}
+
+sub fulldir {
+ my ($topdir, $subdir) = @_;
+ return $subdir =~ /^\// ? $subdir : $topdir.$subdir;
+}
+
+sub terminal_friendly_spawn {
+ my $actionmsg = shift;
+ my $sh = shift;
+ my $quiet = shift;
+ my $minimal = shift;
+ my $output = "";
+ if ($terminal) {
+ my $pty = IO::Pty::Easy->new;
+ $pty->spawn($sh);
+ while ($pty->is_active) {
+ my $data = $pty->read();
+ $output .= $data if defined $data;
+ }
+ $pty->close;
+ } else {
+ $output = qx/$sh 2>&1/;
+ }
+ my $ret = $?;
+ if ($quiet && $ret != 0) {
+ print "$actionmsg\n" if $actionmsg;
+ print STDERR $output;
+ } elsif (!$quiet && (!$minimal || $output)) {
+ print "$actionmsg\n" if $actionmsg;
+ print $output;
+ }
+ return ($ret, $output ? 1 : 0);
+}
+
+sub action {
+ my ($action, $dir, $topdir, $subdir, $force_checkout) = @_;
+ my $fulldir=fulldir($topdir, $subdir);
+ my $checkout_dir;
+
+ $ENV{MR_CONFIG}=$configfiles{$topdir};
+ my $is_checkout=($action eq 'checkout');
+ my $is_update=($action =~ /update/);
+
+ ($ENV{MR_REPO}=$dir) =~ s!/$!!;
+ $ENV{MR_ACTION}=$action;
+
+ foreach my $testname ("skip", "deleted") {
+ next if $force && $testname eq "skip";
+
+ my $testcommand=findcommand($testname, $dir, $topdir, $subdir, $is_checkout);
+
+ if (defined $testcommand) {
+ my $ret=runsh "$testname test", $topdir, $subdir,
+ $testcommand, [$action],
+ sub { system(shift()) };
+ if ($ret != 0) {
+ if (($? & 127) == 2) {
+ print STDERR "mr $action: interrupted\n";
+ return ABORT;
+ }
+ elsif ($? & 127) {
+ print STDERR "mr $action: $testname test received signal ".($? & 127)."\n";
+ return ABORT;
+ }
+ }
+ if ($ret >> 8 == 0) {
+ if ($testname eq "deleted") {
+ if (-d $dir) {
+ print STDERR "mr error: $dir should be deleted yet still exists\n";
+ return FAILED;
+ }
+ }
+ print "mr $action: skip $dir skipped\n" if $verbose;
+ return SKIPPED;
+ }
+ }
+ }
+
+ if ($is_checkout) {
+ $checkout_dir=$dir;
+ if (! $force_checkout) {
+ if (-d $dir) {
+ print "mr $action: $dir already exists, skipping checkout\n" if $verbose;
+ return SKIPPED;
+ }
+
+ $dir=~s/^(.*)\/[^\/]+\/?$/$1/;
+ }
+ }
+ elsif ($is_update) {
+ if (! -d $dir) {
+ return action("checkout", $dir, $topdir, $subdir);
+ }
+ }
+
+ my $command=findcommand($action, $dir, $topdir, $subdir, $is_checkout);
+
+ if ($is_checkout && ! -d $dir) {
+ print "mr $action: creating parent directory $dir\n" if $verbose;
+ system("mkdir", "-p", $dir);
+ }
+
+ if (! $no_chdir && ! chdir($dir)) {
+ print STDERR "mr $action: failed to chdir to $dir: $!\n";
+ return FAILED;
+ }
+ elsif (! defined $command) {
+ my $vcs=vcs_test(@_);
+ if (! defined $vcs) {
+ print STDERR "mr $action: unknown repository type and no defined $action command for $fulldir\n";
+ return FAILED;
+ }
+ else {
+ print STDERR "mr $action: no defined action for $vcs repository $fulldir, skipping\n" unless $minimal;
+ return SKIPPED;
+ }
+ }
+ else {
+ my $actionmsg;
+ if (! $no_chdir) {
+ $actionmsg="mr $action: $fulldir";
+ }
+ else {
+ my $s=$directory;
+ $s=~s/^\Q$fulldir\E\/?//;
+ $actionmsg="mr $action: $fulldir (in subdir $s)";
+ }
+ print "$actionmsg\n" unless $quiet || $minimal;
+
+ my ($hookret, $hook_out)=hook("pre_$action", $topdir, $subdir);
+ return $hookret if $hookret != OK;
+
+ my ($ret, $out)=runsh $action, $topdir, $subdir,
+ $command, \@ARGV, sub {
+ my $sh=shift;
+ if (!$jobs || $jobs > 1 || $quiet || $minimal) {
+ return terminal_friendly_spawn($actionmsg, $sh, $quiet, $minimal);
+ }
+ else {
+ system($sh);
+ }
+ };
+ if ($ret != 0) {
+ if (($? & 127) == 2) {
+ print STDERR "mr $action: interrupted\n";
+ return ABORT;
+ }
+ elsif ($? & 127) {
+ print STDERR "mr $action: received signal ".($? & 127)."\n";
+ return ABORT;
+ }
+ print STDERR "mr $action: failed ($ret)\n" if $verbose;
+ if ($ret >> 8 != 0) {
+ print STDERR "mr $action: command failed\n";
+ if (-e "$ENV{HOME}/.mrlog" && $action ne 'remember') {
+ # recreate original command line to
+ # remember, and avoid recursing
+ my @orig=@ARGV;
+ @ARGV=('-n', $action, @orig);
+ action("remember", $dir, $topdir, $subdir);
+ @ARGV=@orig;
+ }
+ }
+ elsif ($ret != 0) {
+ print STDERR "mr $action: command died ($ret)\n";
+ }
+ return FAILED;
+ }
+ else {
+ if ($is_checkout && ! -d $dir) {
+ print STDERR "mr $action: $dir missing after checkout\n";;
+ return FAILED;
+ }
+
+ my ($ret, $hook_out)=hook("post_$action", $topdir, $subdir);
+ return $ret if $ret != OK;
+
+ if ($is_checkout || $is_update) {
+ if ($is_checkout && ! $no_chdir) {
+ if (! chdir($checkout_dir)) {
+ print STDERR "mr $action: failed to chdir to $checkout_dir: $!\n";
+ return FAILED;
+ }
+ }
+ my ($ret, $hook_out)=hook("fixups", $topdir, $subdir);
+ return $ret if $ret != OK;
+ }
+
+ return (OK, $out || $hook_out);
+ }
+ }
+}
+
+sub hook {
+ my ($hook, $topdir, $subdir) = @_;
+
+ my $command=$config{$topdir}{$subdir}{$hook};
+ return OK unless defined $command;
+ my ($ret,$out)=runsh $hook, $topdir, $subdir, $command, [], sub {
+ my $sh=shift;
+ if (!$jobs || $jobs > 1 || $quiet || $minimal) {
+ return terminal_friendly_spawn(undef, $sh, $quiet, $minimal);
+ }
+ else {
+ system($sh);
+ }
+ };
+ if ($ret != 0) {
+ if (($? & 127) == 2) {
+ print STDERR "mr $hook: interrupted\n";
+ return ABORT;
+ }
+ elsif ($? & 127) {
+ print STDERR "mr $hook: received signal ".($? & 127)."\n";
+ return ABORT;
+ }
+ else {
+ return FAILED;
+ }
+ }
+
+ return (OK, $out);
+}
+
+# run actions on multiple repos, in parallel
+sub mrs {
+ my $action=shift;
+ my @repos=@_;
+
+ $| = 1;
+ my @active;
+ my @fhs;
+ my @out;
+ my $running=0;
+ while (@fhs or @repos) {
+ while ((!$jobs || $running < $jobs) && @repos) {
+ $running++;
+ my $repo = shift @repos;
+ pipe(my $outfh, CHILD_STDOUT);
+ pipe(my $errfh, CHILD_STDERR);
+ my $pid;
+ unless ($pid = fork) {
+ die "mr $action: cannot fork: $!" unless defined $pid;
+ open(STDOUT, ">&CHILD_STDOUT") || die "mr $action cannot reopen stdout: $!";
+ open(STDERR, ">&CHILD_STDERR") || die "mr $action cannot reopen stderr: $!";
+ close CHILD_STDOUT;
+ close CHILD_STDERR;
+ close $outfh;
+ close $errfh;
+ exit +(action($action, @$repo))[0];
+ }
+ close CHILD_STDOUT;
+ close CHILD_STDERR;
+ push @active, [$pid, $repo];
+ push @fhs, [$outfh, $errfh];
+ push @out, ['', ''];
+ }
+ my ($rin, $rout) = ('','');
+ my $nfound;
+ foreach my $fh (@fhs) {
+ next unless defined $fh;
+ vec($rin, fileno($fh->[0]), 1) = 1 if defined $fh->[0];
+ vec($rin, fileno($fh->[1]), 1) = 1 if defined $fh->[1];
+ }
+ $nfound = select($rout=$rin, undef, undef, 1);
+ foreach my $channel (0, 1) {
+ foreach my $i (0..$#fhs) {
+ next unless defined $fhs[$i];
+ my $fh = $fhs[$i][$channel];
+ next unless defined $fh;
+ if (vec($rout, fileno($fh), 1) == 1) {
+ my $r = '';
+ if (sysread($fh, $r, 1024) == 0) {
+ close($fh);
+ $fhs[$i][$channel] = undef;
+ if (! defined $fhs[$i][0] &&
+ ! defined $fhs[$i][1]) {
+ waitpid($active[$i][0], 0);
+ print STDOUT $out[$i][0];
+ print STDERR $out[$i][1];
+ record($active[$i][1], $? >> 8, $out[$i][0] || $out[$i][1]);
+ splice(@fhs, $i, 1);
+ splice(@active, $i, 1);
+ splice(@out, $i, 1);
+ $running--;
+ }
+ }
+ $out[$i][$channel] .= $r;
+ }
+ }
+ }
+ }
+}
+
+sub record {
+ my $dir=shift()->[0];
+ my $ret=shift;
+ my $out=shift;
+
+ if ($ret == OK) {
+ push @ok, $dir;
+ print "\n" unless $quiet || ($minimal && !$out);
+ }
+ elsif ($ret == FAILED) {
+ if ($interactive) {
+ chdir($dir) unless $no_chdir;
+ print STDERR "mr: Starting interactive shell. Exit shell to continue.\n";
+ system((getpwuid($<))[8], "-i");
+ }
+ push @failed, $dir;
+ print "\n";
+ }
+ elsif ($ret == SKIPPED) {
+ push @skipped, $dir;
+ }
+ elsif ($ret == ABORT) {
+ exit 1;
+ }
+ else {
+ die "unknown exit status $ret";
+ }
+}
+
+sub showstats {
+ my $action=shift;
+ if (! @ok && ! @failed && ! @skipped) {
+ die "mr $action: no repositories found to work on\n";
+ }
+ print "mr $action: finished (".join("; ",
+ showstat($#ok+1, "ok", "ok"),
+ showstat($#failed+1, "failed", "failed"),
+ showstat($#skipped+1, "skipped", "skipped"),
+ ).")\n" unless $quiet || $minimal;
+ if ($stats) {
+ if (@skipped) {
+ print "mr $action: (skipped: ".join(" ", @skipped).")\n" unless $quiet || $minimal;
+ }
+ if (@failed) {
+ print STDERR "mr $action: (failed: ".join(" ", @failed).")\n";
+ }
+ }
+}
+
+sub showstat {
+ my $count=shift;
+ my $singular=shift;
+ my $plural=shift;
+ if ($count) {
+ return "$count ".($count > 1 ? $plural : $singular);
+ }
+ return;
+}
+
+# an ordered list of repos
+sub repolist {
+ my @list;
+ foreach my $topdir (sort keys %config) {
+ foreach my $subdir (sort keys %{$config{$topdir}}) {
+ push @list, {
+ topdir => $topdir,
+ subdir => $subdir,
+ order => $config{$topdir}{$subdir}{order},
+ };
+ }
+ }
+ return sort {
+ $a->{order} <=> $b->{order}
+ ||
+ $a->{topdir} cmp $b->{topdir}
+ ||
+ $a->{subdir} cmp $b->{subdir}
+ } @list;
+}
+
+sub repodir {
+ my $repo=shift;
+ my $topdir=$repo->{topdir};
+ my $subdir=$repo->{subdir};
+ my $ret=($subdir =~/^\//) ? $subdir : $topdir.$subdir;
+ $ret=~s/\/\.$//;
+ return $ret;
+}
+
+# Figure out which repos to act on. Returns a list of array refs
+# in the format:
+#
+# [ "$full_repo_path/", "$mr_config_path/", $section_header ]
+sub selectrepos {
+ my @repos;
+ foreach my $repo (repolist()) {
+ my $topdir=$repo->{topdir};
+ my $subdir=$repo->{subdir};
+
+ next if $subdir eq 'DEFAULT';
+ my $dir=repodir($repo);
+ my $d=$directory;
+ $dir.="/" unless $dir=~/\/$/;
+ $d.="/" unless $d=~/\/$/;
+ next if $dir ne $d && $dir !~ /^\Q$d\E/;
+ if (defined $max_depth) {
+ my @a=split('/', $dir);
+ my @b=split('/', $d);
+ do { } while (@a && @b && shift(@a) eq shift(@b));
+ next if @a > $max_depth || @b > $max_depth;
+ }
+ push @repos, [$dir, $topdir, $subdir];
+ }
+ if (! @repos) {
+ # fallback to find a leaf repo
+ foreach my $repo (reverse repolist()) {
+ my $topdir=$repo->{topdir};
+ my $subdir=$repo->{subdir};
+
+ next if $subdir eq 'DEFAULT';
+ my $dir=repodir($repo);
+ my $d=$directory;
+ $dir.="/" unless $dir=~/\/$/;
+ $d.="/" unless $d=~/\/$/;
+ if ($d=~/^\Q$dir\E/) {
+ push @repos, [$dir, $topdir, $subdir];
+ last;
+ }
+ }
+ $no_chdir=1;
+ }
+ return @repos;
+}
+
+sub expandenv {
+ my $val=shift;
+
+
+ if ($val=~/\$/) {
+ $val=`echo "$val"`;
+ chomp $val;
+ }
+
+ return $val;
+}
+
+my %trusted;
+sub is_trusted_config {
+ my $config=shift; # must be abs_pathed already
+
+ # We always trust ~/.mrconfig.
+ return 1 if $config eq abs_path($HOME_MR_CONFIG);
+
+ return 1 if $trust_all;
+
+ my $trustfile=$ENV{HOME}."/.mrtrust";
+
+ if (! %trusted) {
+ $trusted{$HOME_MR_CONFIG}=1;
+ if (open (TRUST, "<", $trustfile)) {
+ while (<TRUST>) {
+ chomp;
+ s/^~\//$ENV{HOME}\//;
+ my $d=abs_path($_);
+ $trusted{$d}=1 if defined $d;
+ }
+ close TRUST;
+ }
+ }
+
+ return $trusted{$config};
+}
+
+
+sub is_trusted_repo {
+ my $repo=shift;
+
+ # Tightly limit what is allowed in a repo name.
+ # No ../, no absolute paths, and no unusual filenames
+ # that might try to escape to the shell.
+ return $repo =~ /^[-_.+\/A-Za-z0-9]+$/ &&
+ $repo !~ /\.\./ && $repo !~ /^\//;
+}
+
+sub is_trusted_checkout {
+ my $command=shift;
+
+ # To determine if the command is safe, compare it with the
+ # *_trusted_checkout config settings. Those settings are
+ # templates for allowed commands, so make sure that each word
+ # of the command matches the corresponding word of the template.
+
+ my @words;
+ foreach my $word (split(' ', $command)) {
+ # strip quoting
+ if ($word=~/^'(.*)'$/) {
+ $word=$1;
+ }
+ elsif ($word=~/^"(.*)"$/) {
+ $word=$1;
+ }
+
+ push @words, $word;
+ }
+
+ foreach my $key (grep { /_trusted_checkout$/ }
+ keys %{$config{''}{DEFAULT}}) {
+ my @twords=split(' ', $config{''}{DEFAULT}{$key});
+ next if @words > @twords;
+
+ my $match=1;
+ my $url;
+ for (my $c=0; $c < @twords && $match; $c++) {
+ if ($twords[$c] eq '$url') {
+ # Match all the typical characters found in
+ # urls, plus @ which svn can use. Note
+ # that the "url" might also be a local
+ # directory.
+ $match=(
+ defined $words[$c] &&
+ $words[$c] =~ /^[-_.+:@\/A-Za-z0-9]+$/
+ );
+ $url=$words[$c];
+ }
+ elsif ($twords[$c] eq '$repo') {
+ # If a repo is not specified, assume it
+ # will be the last path component of the
+ # url, or something derived from it, and
+ # check that.
+ if (! defined $words[$c] && defined $url) {
+ ($words[$c])=$url=~/\/([^\/]+)\/?$/;
+ }
+
+ $match=(
+ defined $words[$c] &&
+ is_trusted_repo($words[$c])
+ );
+ }
+ elsif (defined $words[$c] && $words[$c]=~/^($twords[$c])$/) {
+ $match=1;
+ }
+ else {
+ $match=0;
+ }
+ }
+ return 1 if $match;
+ }
+
+ return 0;
+}
+
+my %loaded;
+sub loadconfig {
+ my $f=shift;
+ my $dir=shift;
+ my $bootstrap_src=shift;
projects / code / myrepos.git / blobdiff