]> git.madduck.net Git - code/myrepos.git/blob - mr

madduck's git repository

Every one of the projects in this repository is available at the canonical URL git://git.madduck.net/madduck/pub/<projectpath> — see each project's metadata for the exact URL.

All patches and comments are welcome. Please squash your changes to logical commits before using git-format-patch and git-send-email to patches@git.madduck.net. If you'd read over the Git project's submission guidelines and adhered to them, I'd be especially grateful.

SSH access, as well as push access can be individually arranged.

If you use my repositories frequently, consider adding the following snippet to ~/.gitconfig and using the third clone URL listed for each project:

[url "git://git.madduck.net/madduck/"]
  insteadOf = madduck:

bugfixes developed while writing blog post
[code/myrepos.git] / mr
1 #!/usr/bin/perl
2
3 =head1 NAME
4
5 mr - a Multiple Repository management tool
6
7 =head1 SYNOPSIS
8
9 B<mr> [options] checkout
10
11 B<mr> [options] update
12
13 B<mr> [options] status
14
15 B<mr> [options] commit [-m "message"]
16
17 B<mr> [options] diff
18
19 B<mr> [options] log
20
21 B<mr> [options] register repository
22
23 B<mr> [options] config section [parameter=[value] ...]
24
25 B<mr> [options] action [params ...]
26
27 =head1 DESCRIPTION
28
29 B<mr> is a Multiple Repository management tool. It allows you to register a
30 set of repositories in a .mrconfig file, and then checkout, update, or
31 perform other actions on the repositories as if they were one big
32 respository.
33
34 Any mix of revision control systems can be used with B<mr>, and you can
35 define arbitrary actions for commands like "update", "checkout", or "commit".
36
37 B<mr> cds into and operates on all registered repsitories at or below your
38 working directory. Or, if you are in a subdirectory of a repository that
39 contains no other registered repositories, it will stay in that directory,
40 and work on only that repository,
41
42 The predefined commands should be fairly familiar to users of any revision
43 control system:
44
45 =over 4
46
47 =item checkout (or co)
48
49 Checks out any repositories that are not already checked out.
50
51 =item update
52
53 Updates each repository from its configured remote repository.
54
55 If a repository isn't checked out yet, it will first check it out.
56
57 =item status
58
59 Displays a status report for each repository, showing what
60 uncommitted changes are present in the repository.
61
62 =item commit (or ci)
63
64 Commits changes to each repository. (By default, changes are pushed to the
65 remote repository too, when using distributed systems like git.)
66
67 The optional -m parameter allows specifying a commit message.
68
69 =item diff
70
71 Show a diff of uncommitted changes.
72
73 =item log
74
75 Show the commit log.
76
77 =item list (or ls)
78
79 List the repositories that mr will act on.
80
81 =item register
82
83 The next parameter is the directory of an existing repository. The
84 repository will be registered in the mrconfig file.
85
86 =item config
87
88 Modifies the mrconfig file. The next parameter is the name of the section
89 to add or modify, and it is followed by one or more instances of
90 "parameter=value". Use "parameter=" to remove a parameter. 
91
92 For example, to add (or edit) a repository in src/foo:
93
94   mr config src/foo checkout="svn co svn://example.com/foo/trunk foo"
95
96 =item help
97
98 Displays this help.
99
100 =back
101
102 Actions can be abbreviated to any unambiguous subsctring, so
103 "mr st" is equivilant to "mr status", and "mr up" is equivilant to "mr
104 update"
105
106 Additional parameters can be passed to most commands, and are passed on
107 unchanged to the underlying revision control system. This is mostly useful
108 if the repositories mr will act on all use the same revision control
109 system.
110
111 =head1 OPTIONS
112
113 =over 4
114
115 =item -d directory
116
117 Specifies the topmost directory that B<mr> should work in. The default is
118 the current working directory.
119
120 =item -c mrconfig
121
122 Use the specified mrconfig file, instead of looking for one in your home
123 directory.
124
125 =item -v
126
127 Be verbose.
128
129 =back
130
131 =head1 FILES
132
133 B<mr> is configured by .mrconfig files. It starts by reading the .mrconfig
134 file in your home directory, and this can in turn chain load .mrconfig files
135 from repositories.
136
137 Here is an example .mrconfig file:
138
139   [src]
140   checkout = svn co svn://svn.example.com/src/trunk src
141   chain = true
142
143   [src/linux-2.6]
144   checkout = git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
145
146 The .mrconfig file uses a variant of the INI file format. Lines starting with
147 "#" are comments. Lines ending with "\" are continued on to the next line.
148
149 The "DEFAULT" section allows setting default values for the sections that
150 come after it.
151
152 The "ALIAS" section allows adding aliases for actions. Each parameter
153 is an alias, and its value is the action to use.
154
155 All other sections add repositories. The section header specifies the
156 directory where the repository is located. This is relative to the directory
157 that contains the mrconfig file, but you can also choose to use absolute
158 paths.
159
160 Within a section, each parameter defines a shell command to run to handle a
161 given action. mr contains default handlers for the "update", "status", and
162 "commit" actions, so normally you only need to specify what to do for
163 "checkout".
164
165 Note that these shell commands are run in a "set -e" shell
166 environment, where any additional parameters you pass are available in
167 "$@". The "checkout" command is run in the parent of the repository
168 directory, since the repository isn't checked out yet. All other commands
169 are run inside the repository, though not necessarily at the top of it.
170 The "MR_REPO" environment variable is set to the path to the top of the
171 repository.
172
173 A few parameters have special meanings:
174
175 =over 4
176
177 =item skip
178
179 If the "skip" parameter is set and its command returns nonzero, then B<mr>
180 will skip acting on that repository.
181
182 =item chain
183
184 If the "chain" parameter is set and its command returns nonzero, then B<mr>
185 will try to load a .mrconfig file from the root of the repository. (You
186 should avoid chaining from repositories with untrusted committers.)
187
188 =item deleted
189
190 If the "deleted" parameter is set and its command returns nonzero, then
191 B<mr> will treat the repository as deleted. It won't ever actually delete
192 the repository, but it will warn if it sees the repsoitory's directory.
193 This is useful when one mrconfig file is shared amoung multiple machines,
194 to keep track of and remember to delete old repositories.
195
196 =item lib
197
198 The "lib" parameter can specify some shell code that will be run before each
199 command, this can be a useful way to define shell functions for other commands
200 to use.
201
202 =back
203
204 =head1 AUTHOR
205
206 Copyright 2007 Joey Hess <joey@kitenet.net>
207
208 Licensed under the GNU GPL version 2 or higher.
209
210 http://kitenet.net/~joey/code/mr/
211
212 =cut
213
214 use warnings;
215 use strict;
216 use Getopt::Long;
217 use Cwd qw(getcwd abs_path);
218
219 my $directory=getcwd();
220 my $config="$ENV{HOME}/.mrconfig";
221 my $verbose=0;
222 my %config;
223 my %knownactions;
224 my %alias;
225
226 Getopt::Long::Configure("no_permute");
227 my $result=GetOptions(
228         "d|directory=s" => sub { $directory=abs_path($_[1]) },
229         "c|config=s" => \$config,
230         "verbose" => \$verbose,
231 );
232 if (! $result || @ARGV < 1) {
233         die("Usage: mr [-d directory] action [params ...]\n".
234             "(Use mr help for man page.)\n");
235
236 }
237
238 loadconfig(\*DATA);
239 loadconfig($config);
240 #use Data::Dumper;
241 #print Dumper(\%config);
242
243 eval {
244         use FindBin qw($Bin $Script);
245         $ENV{MR_PATH}=$Bin."/".$Script;
246 };
247
248 # alias expansion and command stemming
249 my $action=shift @ARGV;
250 if (exists $alias{$action}) {
251         $action=$alias{$action};
252 }
253 if (! exists $knownactions{$action}) {
254         my @matches = grep { /^\Q$action\E/ }
255                 keys %knownactions, keys %alias;
256         if (@matches == 1) {
257                 $action=$matches[0];
258         }
259         elsif (@matches == 0) {
260                 die "mr: unknown action \"$action\" (known actions: ".
261                         join(", ", sort keys %knownactions).")\n";
262         }
263         else {
264                 die "mr: ambiguous action \"$action\" (matches: ".
265                         join(", ", @matches).")\n";
266         }
267 }
268
269 if ($action eq 'help') {
270         exec($config{''}{DEFAULT}{$action}) || die "exec: $!";
271 }
272 elsif ($action eq 'config') {
273         if (@ARGV < 2) {
274                 die "mr config: not enough parameters\n";
275         }
276         my $section=shift;
277         if ($section=~/^\//) {
278                 # try to convert to a path relative to $config's dir
279                 my ($dir)=$config=~/^(.*\/)[^\/]+$/;
280                 if ($section=~/^\Q$dir\E(.*)/) {
281                         $section=$1;
282                 }
283         }
284         my %fields;
285         foreach (@ARGV) {
286                 if (/^([^=]+)=(.*)$/) {
287                         $fields{$1}=$2;
288                 }
289                 else {
290                         die "mr config: expected parameter=value, not \"$_\"\n";
291                 }
292         }
293         modifyconfig($config, $section, %fields);
294         exit 0;
295 }
296 elsif ($action eq 'register') {
297         my $command="set -e; ".$config{''}{DEFAULT}{lib}."\n".
298                 "my_action(){ $config{''}{DEFAULT}{$action}\n }; my_action ".
299                 join(" ", map { s/\//\/\//g; s/"/\"/g; '"'.$_.'"' } @ARGV);
300         print STDERR "mr $action: running >>$command<<\n" if $verbose;
301         exec($command) || die "exec: $!";
302 }
303
304 # work out what repos to act on
305 my @repos;
306 my $nochdir=0;
307 foreach my $topdir (sort keys %config) {
308         foreach my $subdir (sort keys %{$config{$topdir}}) {
309                 next if $subdir eq 'DEFAULT';
310                 my $dir=($subdir =~/^\//) ? $subdir : $topdir.$subdir;
311                 my $d=$directory;
312                 $dir.="/" unless $dir=~/\/$/;
313                 $d.="/" unless $d=~/\/$/;
314                 next if $dir ne $directory && $dir !~ /^\Q$directory\E/;
315                 push @repos, [$dir, $topdir, $subdir];
316         }
317 }
318 if (! @repos) {
319         # fallback to find a leaf repo
320         LEAF: foreach my $topdir (reverse sort keys %config) {
321                 foreach my $subdir (reverse sort keys %{$config{$topdir}}) {
322                         next if $subdir eq 'DEFAULT';
323                         my $dir=($subdir =~/^\//) ? $subdir : $topdir.$subdir;
324                         my $d=$directory;
325                         $dir.="/" unless $dir=~/\/$/;
326                         $d.="/" unless $d=~/\/$/;
327                         if ($d=~/^\Q$dir\E/) {
328                                 push @repos, [$dir, $topdir, $subdir];
329                                 last LEAF;
330                         }
331                 }
332         }
333         $nochdir=1;
334 }
335
336 my (@failed, @successful, @skipped);
337 foreach my $repo (@repos) {
338         action($action, @$repo);
339 }
340
341 sub action {
342         my ($action, $dir, $topdir, $subdir) = @_;
343         
344         my $lib= exists $config{$topdir}{$subdir}{lib} ?
345                         $config{$topdir}{$subdir}{lib}."\n" : "";
346
347         if (exists $config{$topdir}{$subdir}{deleted}) {
348                 if (! -d $dir) {
349                         return;
350                 }
351                 else {
352                         my $test="set -e;".$lib.$config{$topdir}{$subdir}{deleted};
353                         print "mr $action: running deleted test >>$test<<\n" if $verbose;
354                         my $ret=system($test);
355                         if ($ret >> 8 == 0) {
356                                 print STDERR "mr error: $dir should be deleted yet still exists\n\n";
357                                 push @failed, $dir;
358                                 return;
359                         }
360                 }
361         }
362
363         if ($action eq 'checkout') {
364                 if (-d $dir) {
365                         print "mr $action: $dir already exists, skipping checkout\n" if $verbose;
366                         push @skipped, $dir;
367                         return;
368                 }
369                 $dir=~s/^(.*)\/[^\/]+\/?$/$1/;
370         }
371         elsif ($action eq 'update') {
372                 if (! -d $dir) {
373                         return action("checkout", $dir, $topdir, $subdir);
374                 }
375         }
376         
377         $ENV{MR_REPO}=$dir;
378
379         if (exists $config{$topdir}{$subdir}{skip}) {
380                 my $test="set -e;".$lib.$config{$topdir}{$subdir}{skip};
381                 print "mr $action: running skip test >>$test<<\n" if $verbose;
382                 my $ret=system($test);
383                 if ($ret >> 8 == 0) {
384                         print "mr $action: $dir skipped per config file\n" if $verbose;
385                         push @skipped, $dir;
386                         return;
387                 }
388         }
389         
390         if (! $nochdir && ! chdir($dir)) {
391                 print STDERR "mr $action: failed to chdir to $dir: $!\n";
392                 push @failed, $dir;
393         }
394         elsif (! exists $config{$topdir}{$subdir}{$action}) {
395                 print STDERR "mr $action: no defined $action command for $topdir$subdir, skipping\n";
396                 push @skipped, $dir;
397         }
398         else {
399                 if (! $nochdir) {
400                         print "mr $action: $topdir$subdir\n";
401                 }
402                 else {
403                         print "mr $action: $topdir$subdir (in subdir $directory)\n";
404                 }
405                 my $command="set -e; ".$lib.
406                         "my_action(){ $config{$topdir}{$subdir}{$action}\n }; my_action ".
407                         join(" ", map { s/\//\/\//g; s/"/\"/g; '"'.$_.'"' } @ARGV);
408                 print STDERR "mr $action: running >>$command<<\n" if $verbose;
409                 my $ret=system($command);
410                 if ($ret != 0) {
411                         print STDERR "mr $action: failed ($ret)\n" if $verbose;
412                         push @failed, $dir;
413                         if ($ret >> 8 != 0) {
414                                 print STDERR "mr $action: command failed\n";
415                         }
416                         elsif ($ret != 0) {
417                                 print STDERR "mr $action: command died ($ret)\n";
418                         }
419                 }
420                 else {
421                         push @successful, $dir;
422                 }
423
424                 print "\n";
425         }
426 }
427
428 sub showstat {
429         my $count=shift;
430         my $singular=shift;
431         my $plural=shift;
432         if ($count) {
433                 return "$count ".($count > 1 ? $plural : $singular);
434         }
435         return;
436 }
437 if (! @successful && ! @failed && ! @skipped) {
438         die "mr $action: no repositories found to work on\n";
439 }
440 print "mr $action: finished (".join("; ",
441         showstat($#successful+1, "successful", "successful"),
442         showstat($#failed+1, "failed", "failed"),
443         showstat($#skipped+1, "skipped", "skipped"),
444 ).")\n";
445 if (@failed) {
446         exit 1;
447 }
448 elsif (! @successful && @skipped) {
449         exit 1;
450 }
451 exit 0;
452
453 my %loaded;
454 sub loadconfig {
455         my $f=shift;
456
457         my @toload;
458
459         my $in;
460         my $dir;
461         if (ref $f eq 'GLOB') {
462                 $in=$f; 
463                 $dir="";
464         }
465         else {
466                 if (! -e $f) {
467                         return;
468                 }
469
470                 my $absf=abs_path($f);
471                 if ($loaded{$absf}) {
472                         return;
473                 }
474                 $loaded{$absf}=1;
475
476                 print "mr: loading config $f\n" if $verbose;
477                 open($in, "<", $f) || die "mr: open $f: $!\n";
478                 ($dir)=$f=~/^(.*\/)[^\/]+$/;
479                 if (! defined $dir) {
480                         $dir=".";
481                 }
482                 $dir=abs_path($dir)."/";
483
484                 # copy in defaults from first parent
485                 my $parent=$dir;
486                 while ($parent=~s/^(.*)\/[^\/]+\/?$/$1/) {
487                         if (exists $config{$parent} &&
488                             exists $config{$parent}{DEFAULT}) {
489                                 $config{$dir}{DEFAULT}={ %{$config{$parent}{DEFAULT}} };
490                                 last;
491                         }
492                 }
493         }
494
495         my $section;
496         while (<$in>) {
497                 chomp;
498                 next if /^\s*\#/ || /^\s*$/;
499                 if (/^\s*\[([^\]]*)\]\s*$/) {
500                         $section=$1;
501                 }
502                 elsif (/^\s*(\w+)\s*=\s*(.*)/) {
503                         my $parameter=$1;
504                         my $value=$2;
505
506                         # continuation line
507                         while ($value=~/(.*)\\$/s) {
508                                 $value=$1."\n".<$in>;
509                                 chomp $value;
510                         }
511
512                         if (! defined $section) {
513                                 die "$f line $.: parameter ($parameter) not in section\n";
514                         }
515                         if ($section ne 'ALIAS' &&
516                             ! exists $config{$dir}{$section} &&
517                             exists $config{$dir}{DEFAULT}) {
518                                 # copy in defaults
519                                 $config{$dir}{$section}={ %{$config{$dir}{DEFAULT}} };
520                         }
521                         if ($section eq 'ALIAS') {
522                                 $alias{$parameter}=$value;
523                         }
524                         elsif ($parameter eq 'lib') {
525                                 $config{$dir}{$section}{lib}.=$value."\n";
526                         }
527                         else {
528                                 $config{$dir}{$section}{$parameter}=$value;
529                                 $knownactions{$parameter}=1;
530                                 if ($parameter eq 'chain' &&
531                                     length $dir && $section ne "DEFAULT" &&
532                                     -e $dir.$section."/.mrconfig" &&
533                                     system($value) >> 8 == 0) {
534                                         push @toload, $dir.$section."/.mrconfig";
535                                 }
536                         }
537                 }
538                 else {
539                         die "$f line $.: parse error\n";
540                 }
541         }
542         close $in;
543
544         foreach (@toload) {
545                 loadconfig($_);
546         }
547 }
548
549 sub modifyconfig {
550         my $f=shift;
551         # the section to modify or add
552         my $targetsection=shift;
553         # fields to change in the section
554         # To remove a field, set its value to "".
555         my %changefields=@_;
556
557         my @lines;
558         my @out;
559
560         if (-e $f) {
561                 open(my $in, "<", $f) || die "mr: open $f: $!\n";
562                 @lines=<$in>;
563                 close $in;
564         }
565
566         my $addfields=sub {
567                 my @blanks;
568                 while ($out[$#out] =~ /^\s*$/) {
569                         unshift @blanks, pop @out;
570                 }
571                 foreach my $field (sort keys %changefields) {
572                         if (length $changefields{$field}) {
573                                 push @out, "$field = $changefields{$field}\n";
574                         }
575                 }
576                 push @out, @blanks;
577         };
578
579         my $section;
580         while (@lines) {
581                 $_=shift(@lines);
582
583                 if (/^\s*\#/ || /^\s*$/) {
584                         push @out, $_;
585                 }
586                 elsif (/^\s*\[([^\]]*)\]\s*$/) {
587                         if (defined $section && 
588                             $section eq $targetsection) {
589                                 $addfields->();
590                         }
591
592                         $section=$1;
593
594                         push @out, $_;
595                 }
596                 elsif (/^\s*(\w+)\s*=\s(.*)/) {
597                         my $parameter=$1;
598                         my $value=$2;
599
600                         # continuation line
601                         while ($value=~/(.*\\)$/s) {
602                                 $value=$1."\n".shift(@lines);
603                                 chomp $value;
604                         }
605
606                         if ($section eq $targetsection) {
607                                 if (exists $changefields{$parameter}) {
608                                         if (length $changefields{$parameter}) {
609                                                 $value=$changefields{$parameter};
610                                         }
611                                         delete $changefields{$parameter};
612                                 }
613                         }
614
615                         push @out, "$parameter = $value\n";
616                 }
617         }
618
619         if (defined $section && 
620             $section eq $targetsection) {
621                         $addfields->();
622         }
623         elsif (%changefields) {
624                 push @out, "\n[$targetsection]\n";
625                 foreach my $field (sort keys %changefields) {
626                         if (length $changefields{$field}) {
627                                 push @out, "$field = $changefields{$field}\n";
628                         }
629                 }
630         }
631
632         open(my $out, ">", $f) || die "mr: write $f: $!\n";
633         print $out @out;
634         close $out;     
635 }
636
637 # Finally, some useful actions that mr knows about by default.
638 # These can be overridden in ~/.mrconfig.
639 __DATA__
640 [ALIAS]
641         co = checkout
642         ci = commit
643         ls = list
644
645 [DEFAULT]
646 lib =                                                   \
647         error() {                                       \
648                 echo "mr: $@" >&2                       \
649                 exit 1                                  \
650         }
651
652 update =                                                \
653         if [ -d "$MR_REPO"/.svn ]; then                 \
654                 svn update "$@"                         \
655         elif [ -d "$MR_REPO"/.git ]; then               \
656                 git pull origin master "$@"             \
657         elif [ -d "$MR_REPO"/.bzr ]; then               \
658                 bzr merge "$@"                          \
659         elif [ -d "$MR_REPO"/CVS ]; then                \
660                 cvs update "$@"                         \
661         else                                            \
662                 error "unknown repo type"               \
663         fi
664 status =                                                \
665         if [ -d "$MR_REPO"/.svn ]; then                 \
666                 svn status "$@"                         \
667         elif [ -d "$MR_REPO"/.git ]; then               \
668                 git status "$@" || true                 \
669         elif [ -d "$MR_REPO"/.bzr ]; then               \
670                 bzr status "$@"                         \
671         elif [ -d "$MR_REPO"/CVS ]; then                \
672                 cvs status "$@"                         \
673         else                                            \
674                 error "unknown repo type"               \
675         fi
676 commit =                                                \
677         if [ -d "$MR_REPO"/.svn ]; then                 \
678                 svn commit "$@"                         \
679         elif [ -d "$MR_REPO"/.git ]; then               \
680                 git commit -a "$@" && git push --all    \
681         elif [ -d "$MR_REPO"/.bzr ]; then               \
682                 bzr commit "$@" && bzr push             \
683         elif [ -d "$MR_REPO"/CVS ]; then                \
684                 cvs commit "$@"                         \
685         else                                            \
686                 error "unknown repo type"               \
687         fi
688 diff =                                                  \
689         if [ -d "$MR_REPO"/.svn ]; then                 \
690                 svn diff "$@"                           \
691         elif [ -d "$MR_REPO"/.git ]; then               \
692                 git diff "$@"                           \
693         elif [ -d "$MR_REPO"/.bzr ]; then               \
694                 bzr diff "$@"                           \
695         elif [ -d "$MR_REPO"/CVS ]; then                \
696                 cvs diff "$@"                           \
697         else                                            \
698                 error "unknown repo type"               \
699         fi
700 log =                                                   \
701         if [ -d "$MR_REPO"/.svn ]; then                 \
702                 svn log"$@"                             \
703         elif [ -d "$MR_REPO"/.git ]; then               \
704                 git log "$@"                            \
705         elif [ -d "$MR_REPO"/.bzr ]; then               \
706                 bzr log "$@"                            \
707         elif [ -d "$MR_REPO"/CVS ]; then                \
708                 cvs log "$@"                            \
709         else                                            \
710                 error "unknown repo type"               \
711         fi
712 register =                                                              \
713         if [ -z "$1" ]; then                                            \
714                 error "repository directory not specified"              \
715         fi                                                              \
716         cd "$1"                                                         \
717         basedir="$(basename $(pwd))"                                    \
718         if [ -d .svn ]; then                                            \
719                 url=$(svn info . |                                      \
720                       grep -i ^URL: | cut -d ' ' -f 2)                  \
721                 if [ -z "$url" ]; then                                  \
722                         error "cannot determine svn url"                \
723                 fi                                                      \
724                 echo "Registering svn url: $url"                        \
725                 mr config "$(pwd)" checkout="svn co $url $basedir"      \
726         elif [ -d .git ]; then                                          \
727                 url=$(git-config --get remote.origin.url)               \
728                 if [ -z "$url" ]; then                                  \
729                         error "cannot determine git url"                \
730                 fi                                                      \
731                 echo "Registering git url: $url"                        \
732                 mr config "$(pwd)" checkout="git clone $url $basedir"   \
733         else                                                            \
734                 error "unable to register this repo type"               \
735         fi
736 list = true
737 config = 
738 help =                                                  \
739         if [ ! -e "$MR_PATH" ]; then                    \
740                 error "cannot find program path"        \
741         fi                                              \
742         (pod2man -c mr "$MR_PATH" | man -l -) ||        \
743                 error "pod2man or man failed"
744
745 ed = echo "A horse is a horse, of course, of course.."
746 T = echo "I pity the fool."