]> git.madduck.net Git - code/myrepos.git/blob - mr

madduck's git repository

Every one of the projects in this repository is available at the canonical URL git://git.madduck.net/madduck/pub/<projectpath> — see each project's metadata for the exact URL.

All patches and comments are welcome. Please squash your changes to logical commits before using git-format-patch and git-send-email to patches@git.madduck.net. If you'd read over the Git project's submission guidelines and adhered to them, I'd be especially grateful.

SSH access, as well as push access can be individually arranged.

If you use my repositories frequently, consider adding the following snippet to ~/.gitconfig and using the third clone URL listed for each project:

[url "git://git.madduck.net/madduck/"]
  insteadOf = madduck:

implemented mr register
[code/myrepos.git] / mr
1 #!/usr/bin/perl
2
3 =head1 NAME
4
5 mr - a Multiple Repository management tool
6
7 =head1 SYNOPSIS
8
9 B<mr> [options] checkout
10
11 B<mr> [options] update
12
13 B<mr> [options] status
14
15 B<mr> [options] commit [-m "message"]
16
17 B<mr> [options] diff
18
19 B<mr> [options] log
20
21 B<mr> [options] register repository
22
23 B<mr> [options] config section [parameter=[value] ...]
24
25 B<mr> [options] action [params ...]
26
27 =head1 DESCRIPTION
28
29 B<mr> is a Multiple Repository management tool. It allows you to register a
30 set of repositories in a .mrconfig file, and then checkout, update, or
31 perform other actions on the repositories as if they were one big
32 respository.
33
34 Any mix of revision control systems can be used with B<mr>, and you can
35 define arbitrary actions for commands like "update", "checkout", or "commit".
36
37 B<mr> cds into and operates on all registered repsitories at or below your
38 working directory. Or, if you are in a subdirectory of a repository that
39 contains no other registered repositories, it will stay in that directory,
40 and work on only that repository,
41
42 The predefined commands should be fairly familiar to users of any revision
43 control system:
44
45 =over 4
46
47 =item checkout (or co)
48
49 Checks out any repositories that are not already checked out.
50
51 =item update
52
53 Updates each repository from its configured remote repository.
54
55 If a repository isn't checked out yet, it will first check it out.
56
57 =item status
58
59 Displays a status report for each repository, showing what
60 uncommitted changes are present in the repository.
61
62 =item commit (or ci)
63
64 Commits changes to each repository. (By default, changes are pushed to the
65 remote repository too, when using distributed systems like git.)
66
67 The optional -m parameter allows specifying a commit message.
68
69 =item diff
70
71 Show a diff of uncommitted changes.
72
73 =item log
74
75 Show the commit log.
76
77 =item list (or ls)
78
79 List the repositories that mr will act on.
80
81 =item register
82
83 The next parameter is the directory of an existing repository. The
84 repository will be registered in the mrconfig file.
85
86 =item config
87
88 Modifies the mrconfig file. The next parameter is the name of the section
89 to add or modify, and it is followed by one or more instances of
90 "parameter=value". Use "parameter=" to remove a parameter. 
91
92 For example, to register a new svn repository in src/foo:
93
94   mr config src/foo checkout="svn co svn://example.com/foo/trunk"
95
96 =item help
97
98 Displays this help.
99
100 =back
101
102 Actions can be abbreviated to any unambiguous subsctring, so
103 "mr st" is equivilant to "mr status", and "mr up" is equivilant to "mr
104 update"
105
106 Additional parameters can be passed to other commands than "commit", they
107 will be passed on unchanged to the underlying revision control system.
108 This is mostly useful if the repositories mr will act on all use the same
109 revision control system.
110
111 =head1 OPTIONS
112
113 =over 4
114
115 =item -d directory
116
117 Specifies the topmost directory that B<mr> should work in. The default is
118 the current working directory.
119
120 =item -c mrconfig
121
122 Use the specified mrconfig file, instead of looking for one in your home
123 directory.
124
125 =item -v
126
127 Be verbose.
128
129 =back
130
131 =head1 FILES
132
133 B<mr> is configured by .mrconfig files. It starts by reading the .mrconfig
134 file in your home directory, and this can in turn chain load .mrconfig files
135 from repositories.
136
137 Here is an example .mrconfig file:
138
139   [src]
140   checkout = svn co svn://svn.example.com/src/trunk src
141   chain = true
142
143   [src/linux-2.6]
144   checkout = git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
145
146 The .mrconfig file uses a variant of the INI file format. Lines starting with
147 "#" are comments. Lines ending with "\" are continued on to the next line.
148
149 The "DEFAULT" section allows setting default values for the sections that
150 come after it.
151
152 The "ALIAS" section allows adding aliases for actions. Each parameter
153 is an alias, and its value is the action to use.
154
155 All other sections add repositories. The section header specifies the
156 directory where the repository is located. This is relative to the directory
157 that contains the mrconfig file, but you can also choose to use absolute
158 paths.
159
160 Within a section, each parameter defines a shell command to run to handle a
161 given action. mr contains default handlers for the "update", "status", and
162 "commit" actions, so normally you only need to specify what to do for
163 "checkout".
164
165 Note that these shell commands are run in a "set -e" shell
166 environment, where any additional parameters you pass are available in
167 "$@". The "checkout" command is run in the parent of the repository
168 directory, since the repository isn't checked out yet. All other commands
169 are run inside the repository, though not necessarily at the top of it.
170 The "MR_REPO" environment variable is set to the path to the top of the
171 repository.
172
173 A few parameters have special meanings:
174
175 =over 4
176
177 =item skip
178
179 If the "skip" parameter is set and its command returns nonzero, then B<mr>
180 will skip acting on that repository.
181
182 =item chain
183
184 If the "chain" parameter is set and its command returns nonzero, then B<mr>
185 will try to load a .mrconfig file from the root of the repository. (You
186 should avoid chaining from repositories with untrusted committers.)
187
188 =item deleted
189
190 If the "deleted" parameter is set and its command returns nonzero, then
191 B<mr> will treat the repository as deleted. It won't ever actually delete
192 the repository, but it will warn if it sees the repsoitory's directory.
193 This is useful when one mrconfig file is shared amoung multiple machines,
194 to keep track of and remember to delete old repositories.
195
196 =item lib
197
198 The "lib" parameter can specify some shell code that will be run before each
199 command, this can be a useful way to define shell functions for other commands
200 to use.
201
202 =back
203
204 =head1 AUTHOR
205
206 Copyright 2007 Joey Hess <joey@kitenet.net>
207
208 Licensed under the GNU GPL version 2 or higher.
209
210 http://kitenet.net/~joey/code/mr/
211
212 =cut
213
214 use warnings;
215 use strict;
216 use Getopt::Long;
217 use Cwd qw(getcwd abs_path);
218
219 my $directory=getcwd();
220 my $config="$ENV{HOME}/.mrconfig";
221 my $verbose=0;
222 my %config;
223 my %knownactions;
224 my %alias;
225
226 Getopt::Long::Configure("no_permute");
227 my $result=GetOptions(
228         "d|directory=s" => sub { $directory=abs_path($_[1]) },
229         "c|config=s" => \$config,
230         "verbose" => \$verbose,
231 );
232 if (! $result || @ARGV < 1) {
233         die("Usage: mr [-d directory] action [params ...]\n".
234             "(Use mr help for man page.)\n");
235
236 }
237
238 loadconfig(\*DATA);
239 loadconfig($config);
240 #use Data::Dumper;
241 #print Dumper(\%config);
242
243 eval {
244         use FindBin qw($Bin $Script);
245         $ENV{MR_PATH}=$Bin."/".$Script;
246 };
247
248 # alias expansion and command stemming
249 my $action=shift @ARGV;
250 if (exists $alias{$action}) {
251         $action=$alias{$action};
252 }
253 if (! exists $knownactions{$action}) {
254         my @matches = grep { /^\Q$action\E/ }
255                 keys %knownactions, keys %alias;
256         if (@matches == 1) {
257                 $action=$matches[0];
258         }
259         elsif (@matches == 0) {
260                 die "mr: unknown action \"$action\" (known actions: ".
261                         join(", ", sort keys %knownactions).")\n";
262         }
263         else {
264                 die "mr: ambiguous action \"$action\" (matches: ".
265                         join(", ", @matches).")\n";
266         }
267 }
268
269 if ($action eq 'help') {
270         exec($config{''}{DEFAULT}{$action}) || die "exec: $!";
271 }
272 elsif ($action eq 'config') {
273         if (@ARGV < 2) {
274                 die "mr config: not enough parameters\n";
275         }
276         my $section=shift;
277         if ($section=~/^\//) {
278                 # try to convert to a path relative to $config's dir
279                 my ($dir)=$config=~/^(.*\/)[^\/]+$/;
280                 if ($section=~/^\Q$dir\E(.*)/) {
281                         $section=$1;
282                 }
283         }
284         my %fields;
285         foreach (@ARGV) {
286                 if (/^([^=]+)=(.*)$/) {
287                         $fields{$1}=$2;
288                 }
289                 else {
290                         die "mr config: expected parameter=value, not \"$_\"\n";
291                 }
292         }
293         modifyconfig($config, $section, %fields);
294         exit 0;
295 }
296 elsif ($action eq 'register') {
297         my $command="set -e; ".$config{''}{DEFAULT}{lib}."\n".
298                 "my_action(){ $config{''}{DEFAULT}{$action}\n }; my_action ".
299                 join(" ", map { s/\//\/\//g; s/"/\"/g; '"'.$_.'"' } @ARGV);
300         print STDERR "mr $action: running >>$command<<\n" if $verbose;
301         exec($command) || die "exec: $!";
302 }
303
304 # work out what repos to act on
305 my @repos;
306 my $nochdir=0;
307 foreach my $topdir (sort keys %config) {
308         foreach my $subdir (sort keys %{$config{$topdir}}) {
309                 next if $subdir eq 'DEFAULT';
310                 my $dir=($subdir =~/^\//) ? $subdir : $topdir.$subdir;
311                 my $d=$directory;
312                 $dir.="/" unless $dir=~/\/$/;
313                 $d.="/" unless $d=~/\/$/;
314                 next if $dir ne $directory && $dir !~ /^\Q$directory\E/;
315                 push @repos, [$dir, $topdir, $subdir];
316         }
317 }
318 if (! @repos) {
319         # fallback to find a leaf repo
320         LEAF: foreach my $topdir (reverse sort keys %config) {
321                 foreach my $subdir (reverse sort keys %{$config{$topdir}}) {
322                         next if $subdir eq 'DEFAULT';
323                         my $dir=($subdir =~/^\//) ? $subdir : $topdir.$subdir;
324                         my $d=$directory;
325                         $dir.="/" unless $dir=~/\/$/;
326                         $d.="/" unless $d=~/\/$/;
327                         if ($d=~/^\Q$dir\E/) {
328                                 push @repos, [$dir, $topdir, $subdir];
329                                 last LEAF;
330                         }
331                 }
332         }
333         $nochdir=1;
334 }
335
336 my (@failed, @successful, @skipped);
337 foreach my $repo (@repos) {
338         action($action, @$repo);
339 }
340
341 sub action {
342         my ($action, $dir, $topdir, $subdir) = @_;
343         
344         my $lib= exists $config{$topdir}{$subdir}{lib} ?
345                         $config{$topdir}{$subdir}{lib}."\n" : "";
346
347         if (exists $config{$topdir}{$subdir}{deleted}) {
348                 if (! -d $dir) {
349                         next;
350                 }
351                 else {
352                         my $test="set -e;".$lib.$config{$topdir}{$subdir}{deleted};
353                         print "mr $action: running deleted test >>$test<<\n" if $verbose;
354                         my $ret=system($test);
355                         if ($ret >> 8 == 0) {
356                                 print STDERR "mr error: $dir should be deleted yet still exists\n\n";
357                                 push @failed, $dir;
358                                 return;
359                         }
360                 }
361         }
362
363         if ($action eq 'checkout') {
364                 if (-d $dir) {
365                         print "mr $action: $dir already exists, skipping checkout\n" if $verbose;
366                         push @skipped, $dir;
367                         return;
368                 }
369                 $dir=~s/^(.*)\/[^\/]+\/?$/$1/;
370         }
371         elsif ($action eq 'update') {
372                 if (! -d $dir) {
373                         return action("checkout", $dir, $topdir, $subdir);
374                 }
375         }
376         
377         $ENV{MR_REPO}=$dir;
378
379         if (exists $config{$topdir}{$subdir}{skip}) {
380                 my $test="set -e;".$lib.$config{$topdir}{$subdir}{skip};
381                 print "mr $action: running skip test >>$test<<\n" if $verbose;
382                 my $ret=system($test);
383                 if ($ret >> 8 == 0) {
384                         print "mr $action: $dir skipped per config file\n" if $verbose;
385                         push @skipped, $dir;
386                         return;
387                 }
388         }
389         
390         if (! $nochdir && ! chdir($dir)) {
391                 print STDERR "mr $action: failed to chdir to $dir: $!\n";
392                 push @failed, $dir;
393         }
394         elsif (! exists $config{$topdir}{$subdir}{$action}) {
395                 print STDERR "mr $action: no defined $action command for $topdir$subdir, skipping\n";
396                 push @skipped, $dir;
397         }
398         else {
399                 if (! $nochdir) {
400                         print "mr $action: $topdir$subdir\n";
401                 }
402                 else {
403                         print "mr $action: $topdir$subdir (in subdir $directory)\n";
404                 }
405                 my $command="set -e; ".$lib.
406                         "my_action(){ $config{$topdir}{$subdir}{$action}\n }; my_action ".
407                         join(" ", map { s/\//\/\//g; s/"/\"/g; '"'.$_.'"' } @ARGV);
408                 print STDERR "mr $action: running >>$command<<\n" if $verbose;
409                 my $ret=system($command);
410                 if ($ret != 0) {
411                         print STDERR "mr $action: failed ($ret)\n" if $verbose;
412                         push @failed, $dir;
413                         if ($ret >> 8 != 0) {
414                                 print STDERR "mr $action: command failed\n";
415                         }
416                         elsif ($ret != 0) {
417                                 print STDERR "mr $action: command died ($ret)\n";
418                         }
419                 }
420                 else {
421                         push @successful, $dir;
422                 }
423
424                 print "\n";
425         }
426 }
427
428 sub showstat {
429         my $count=shift;
430         my $singular=shift;
431         my $plural=shift;
432         if ($count) {
433                 return "$count ".($count > 1 ? $plural : $singular);
434         }
435         return;
436 }
437 if (! @successful && ! @failed && ! @skipped) {
438         die "mr $action: no repositories found to work on\n";
439 }
440 print "mr $action: finished (".join("; ",
441         showstat($#successful+1, "successful", "successful"),
442         showstat($#failed+1, "failed", "failed"),
443         showstat($#skipped+1, "skipped", "skipped"),
444 ).")\n";
445 if (@failed) {
446         exit 1;
447 }
448 elsif (! @successful && @skipped) {
449         exit 1;
450 }
451 exit 0;
452
453 my %loaded;
454 sub loadconfig {
455         my $f=shift;
456
457         my @toload;
458
459         my $in;
460         my $dir;
461         if (ref $f eq 'GLOB') {
462                 $in=$f; 
463                 $dir="";
464         }
465         else {
466                 my $absf=abs_path($f);
467                 if ($loaded{$absf}) {
468                         return;
469                 }
470                 $loaded{$absf}=1;
471
472                 print "mr: loading config $f\n" if $verbose;
473                 open($in, "<", $f) || die "mr: open $f: $!\n";
474                 ($dir)=$f=~/^(.*\/)[^\/]+$/;
475                 if (! defined $dir) {
476                         $dir=".";
477                 }
478                 $dir=abs_path($dir)."/";
479
480                 # copy in defaults from first parent
481                 my $parent=$dir;
482                 while ($parent=~s/^(.*)\/[^\/]+\/?$/$1/) {
483                         if (exists $config{$parent} &&
484                             exists $config{$parent}{DEFAULT}) {
485                                 $config{$dir}{DEFAULT}={ %{$config{$parent}{DEFAULT}} };
486                                 last;
487                         }
488                 }
489         }
490
491         my $section;
492         while (<$in>) {
493                 chomp;
494                 next if /^\s*\#/ || /^\s*$/;
495                 if (/^\s*\[([^\]]*)\]\s*$/) {
496                         $section=$1;
497                 }
498                 elsif (/^\s*(\w+)\s*=\s*(.*)/) {
499                         my $parameter=$1;
500                         my $value=$2;
501
502                         # continuation line
503                         while ($value=~/(.*)\\$/s) {
504                                 $value=$1."\n".<$in>;
505                                 chomp $value;
506                         }
507
508                         if (! defined $section) {
509                                 die "$f line $.: parameter ($parameter) not in section\n";
510                         }
511                         if ($section ne 'ALIAS' &&
512                             ! exists $config{$dir}{$section} &&
513                             exists $config{$dir}{DEFAULT}) {
514                                 # copy in defaults
515                                 $config{$dir}{$section}={ %{$config{$dir}{DEFAULT}} };
516                         }
517                         if ($section eq 'ALIAS') {
518                                 $alias{$parameter}=$value;
519                         }
520                         elsif ($parameter eq 'lib') {
521                                 $config{$dir}{$section}{lib}.=$value."\n";
522                         }
523                         else {
524                                 $config{$dir}{$section}{$parameter}=$value;
525                                 $knownactions{$parameter}=1;
526                                 if ($parameter eq 'chain' &&
527                                     length $dir && $section ne "DEFAULT" &&
528                                     -e $dir.$section."/.mrconfig" &&
529                                     system($value) >> 8 == 0) {
530                                         push @toload, $dir.$section."/.mrconfig";
531                                 }
532                         }
533                 }
534                 else {
535                         die "$f line $.: parse error\n";
536                 }
537         }
538         close $in;
539
540         foreach (@toload) {
541                 loadconfig($_);
542         }
543 }
544
545 sub modifyconfig {
546         my $f=shift;
547         # the section to modify or add
548         my $targetsection=shift;
549         # fields to change in the section
550         # To remove a field, set its value to "".
551         my %changefields=@_;
552
553         my @lines;
554         if (-e $f) {
555                 open(my $in, "<", $f) || die "mr: open $f: $!\n";
556                 @lines=<$in>;
557                 close $in;
558         }
559
560         my $section;
561         my @out;
562         while (@lines) {
563                 $_=shift(@lines);
564
565                 if (/^\s*\#/ || /^\s*$/) {
566                         push @out, $_;
567                 }
568                 elsif (/^\s*\[([^\]]*)\]\s*$/) {
569                         if (defined $section && 
570                             $section eq $targetsection) {
571                                 my @blanks;
572                                 while ($out[$#out] =~ /^\s*$/) {
573                                         unshift @blanks, pop @out;
574                                 }
575                                 foreach my $field (sort keys %changefields) {
576                                         if (length $changefields{$field}) {
577                                                 push @out, "$field = $changefields{$field}\n";
578                                         }
579                                 }
580                                 push @out, @blanks;
581                         }
582
583                         $section=$1;
584
585                         push @out, $_;
586                 }
587                 elsif (/^\s*(\w+)\s*=\s(.*)/) {
588                         my $parameter=$1;
589                         my $value=$2;
590
591                         # continuation line
592                         while ($value=~/(.*\\)$/s) {
593                                 $value=$1."\n".shift(@lines);
594                                 chomp $value;
595                         }
596
597                         if ($section eq $targetsection) {
598                                 if (exists $changefields{$parameter}) {
599                                         if (length $changefields{$parameter}) {
600                                                 $value=$changefields{$parameter};
601                                         }
602                                         delete $changefields{$parameter};
603                                 }
604                         }
605
606                         push @out, "$parameter = $value\n";
607                 }
608         }
609
610         if (%changefields) {
611                 push @out, "\n[$targetsection]\n";
612                 foreach my $field (sort keys %changefields) {
613                         if (length $changefields{$field}) {
614                                 push @out, "$field = $changefields{$field}\n";
615                         }
616                 }
617         }
618
619         open(my $out, ">", $f) || die "mr: write $f: $!\n";
620         print $out @out;
621         close $out;     
622 }
623
624 # Finally, some useful actions that mr knows about by default.
625 # These can be overridden in ~/.mrconfig.
626 __DATA__
627 [ALIAS]
628         co = checkout
629         ci = commit
630         ls = list
631
632 [DEFAULT]
633 lib =                                                   \
634         error() {                                       \
635                 echo "mr: $@" >&2                       \
636                 exit 1                                  \
637         }
638
639 update =                                                \
640         if [ -d "$MR_REPO"/.svn ]; then                 \
641                 svn update "$@"                         \
642         elif [ -d "$MR_REPO"/.git ]; then               \
643                 git pull origin master "$@"             \
644         elif [ -d "$MR_REPO"/.bzr ]; then               \
645                 bzr merge "$@"                          \
646         elif [ -d "$MR_REPO"/CVS ]; then                \
647                 cvs update "$@"                         \
648         else                                            \
649                 error "unknown repo type"               \
650         fi
651 status =                                                \
652         if [ -d "$MR_REPO"/.svn ]; then                 \
653                 svn status "$@"                         \
654         elif [ -d "$MR_REPO"/.git ]; then               \
655                 git status "$@" || true                 \
656         elif [ -d "$MR_REPO"/.bzr ]; then               \
657                 bzr status "$@"                         \
658         elif [ -d "$MR_REPO"/CVS ]; then                \
659                 cvs status "$@"                         \
660         else                                            \
661                 error "unknown repo type"               \
662         fi
663 commit =                                                \
664         if [ -d "$MR_REPO"/.svn ]; then                 \
665                 svn commit "$@"                         \
666         elif [ -d "$MR_REPO"/.git ]; then               \
667                 git commit -a "$@" && git push --all    \
668         elif [ -d "$MR_REPO"/.bzr ]; then               \
669                 bzr commit "$@" && bzr push             \
670         elif [ -d "$MR_REPO"/CVS ]; then                \
671                 cvs commit "$@"                         \
672         else                                            \
673                 error "unknown repo type"               \
674         fi
675 diff =                                                  \
676         if [ -d "$MR_REPO"/.svn ]; then                 \
677                 svn diff "$@"                           \
678         elif [ -d "$MR_REPO"/.git ]; then               \
679                 git diff "$@"                           \
680         elif [ -d "$MR_REPO"/.bzr ]; then               \
681                 bzr diff "$@"                           \
682         elif [ -d "$MR_REPO"/CVS ]; then                \
683                 cvs diff "$@"                           \
684         else                                            \
685                 error "unknown repo type"               \
686         fi
687 log =                                                   \
688         if [ -d "$MR_REPO"/.svn ]; then                 \
689                 svn log"$@"                             \
690         elif [ -d "$MR_REPO"/.git ]; then               \
691                 git log "$@"                            \
692         elif [ -d "$MR_REPO"/.bzr ]; then               \
693                 bzr log "$@"                            \
694         elif [ -d "$MR_REPO"/CVS ]; then                \
695                 cvs log "$@"                            \
696         else                                            \
697                 error "unknown repo type"               \
698         fi
699 register =                                                              \
700         if [ -z "$1" ]; then                                            \
701                 error "repository directory not specified"              \
702         fi                                                              \
703         cd "$1"                                                         \
704         basedir="$(basename $(pwd))"                                    \
705         if [ -d .svn ]; then                                            \
706                 url=$(svn info . |                                      \
707                       grep -i ^URL: | cut -d ' ' -f 2)                  \
708                 if [ -z "$url" ]; then                                  \
709                         error "cannot determine svn url"                \
710                 fi                                                      \
711                 echo "Found svn url: $url"                              \
712                 mr config "$(pwd)" checkout="svn co $url $basedir"      \
713         elif [ -d .git ]; then                                          \
714                 url=$(git-config --get remote.origin.url)               \
715                 if [ -z "$url" ]; then                                  \
716                         error "cannot determine git url"                \
717                 fi                                                      \
718                 echo "Found git url: $url"                              \
719                 mr config "$(pwd)" checkout="git clone $url $basedir"   \
720         else                                                            \
721                 error "unable to register this repo type"               \
722         fi
723 list = true
724 config = 
725 help =                                                  \
726         if [ ! -e "$MR_PATH" ]; then                    \
727                 error "cannot find program path"        \
728         fi                                              \
729         (pod2man -c mr "$MR_PATH" | man -l -) ||        \
730                 error "pod2man or man failed"
731
732 ed = echo "A horse is a horse, of course, of course.."
733 T = echo "I pity the fool."