[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [E-devel] cvs, servers and stuff.



On Thu, 17 Aug 2006 10:15:13 +0200 Stéphane Bauland <joligardon@gmail.com>
babbled:

> Hisham Mardam Bey wrote:
> > On 8/17/06, Lyle Kempler <term-e1@twistedpath.org> wrote:
> >   
> >> Personally I don't get why we're not creating nightly diffs and a
> >> tarball once a week and letting people use that (it's not exactly
> >> difficult to script). I highly doubt most users care about checkins in
> >> the last few hours, and I suggested doing so I think way back in January
> >> on IRC..
> >>
> >>     
> >
> > Actually they are. You'd be surprised how many people build E (and
> > related stuff) multiple times a day from anoncvs. Again, I agree with
> > the majority that is suggesting the mirrors idea. CVS has its quirks
> > here and there, but its been doing everything we wanted for a long
> > time. I think that having a 2 or 3 anoncvs mirrors should solve our
> > problems.
> 
> I'm agreeing Hisham, the way to got mirrors is better i think... If 
> raster (etc...) wants to change cvs for svn, git or anything else isn't 
> the problems. Having more than one server is always a good idea.
> 
> See ya CodeWaWa :)

i am wanting to open a discussion to solutions. due to peak loads kainx has had
to limit connection #'s - fair enough. this is bad. several anoncvs servers
will help. i think maybe we need to explore other options.

i am loathe to change scm's unless we have to. if we can avoid it - great. i do
NOT deny other scm's have benefits - can do things cvs can't etc. etc. - but
frankly - cvs does everything us developers need. we use only a subset of cvs
too. we don't need the complex features in other scm's - nice though they may
be. the point of this is to try and make anonymous "up to date" source code
access scale better.

it seems that every scm has its positives and negatives. svn and cvs seem
evenly matched performance-wise, give or take. svn increases the local checkout
cost. git makes this even larger, but server-side is very light. of these 3
main contenders none stand out as a major leap forward for anon access.

so we are down to
1. anon cvs mirrors
2. another system (tarball snaps, etc. etc).

personally i would have no problem in a server-side auto-build of tarballs.
what do people think? should we perhaps have the anoncvs server do daily (or
maybe several times per day) builds of packages? not rpm or deb but "make
dist"; ie something like the following run maybe every 4 or 8 hours? once a day?

# how much archive to keep in Kb
MAX=10000000
# date & time snap
DAT=`date '+%F_%T'`
# make the snap dir
mkdir -p /pub/snaps/$DAT
# latest link to automated snarfers can get the latest always
ln -sf /pub/snaps/$DAT /pub/snaps/LATEST
# get cvs
cvs co e17
cd ./e17
# for everything in the cvs tree with a configure.in
for I in `find . -name configure.in -print`; do
  DIR=`dirname $I`
  PRJ=`basename $DIR`
  pushd $DIR
  # just because it has a configure.in does not mean we want a snap - add a
  # magic file to tell us we want to do the snap
  if [ -f ./.DO-SNAP ]; then
    ./autogen.sh
    make dist
    mv $PRJ-*.tar.gz /pub/snaps/$DAT/
  fi
  popd
done
# clean up the checkout
rm -rf ./e17
# clean up old snaps if our archive got to big
SIZ=`du -s /pub/snaps | awk '{printf("%s", $1);}'`
while [ $SIZ -gt $MAX ]; do
  DEL=`/bin/ls /pub/snaps | head -1`
  rm -rf /pub/snaps/$DEL
  SIZ=`du -s /pub/snaps | awk '{printf("%s", $1);}'`
done

We can now make "/pub/snaps" available over http/ftp etc. etc. - hell, even
generate .torrent seed files. maybe stop advertising anoncvs - and encourage
peoples "easy_e17.sh" or "get_e.sh" or emerges etc. to use this repository?
they won't need "autofoo" anymore as the configure etc. will already be
generated. this also catches broken packages - as they simply will refuse to
build a dist tarball and be skipped. well badly broken - if they don't pass
make distcheck - well... we don't catch that (we can add it though) 

what do you say? would people be willing to switch to such snaps?

-- 
------------- Codito, ergo sum - "I code, therefore I am" --------------
The Rasterman (Carsten Haitzler)    raster@rasterman.com
裸好多
Tokyo, Japan (東京 日本)