This post is very old and likely contains information that is no longer accurate and links which no longer work. Proceed with caution.
According to discussions on the FreeBSD-stable list, CVS support for FreeBSD sources is not just dying, it’s practically dead. (I’ll skip over the jokes about Netcraft confirming it…) So I setup an SVN mirror of the FreeBSD base and ports repositories and documented the process a little. I did this mainly because there are a lot of servers in two locations that need frequent access to the base and ports repositories, and there are currently cvsup mirrors running (one at each place) to keep the load off the upstream servers and for faster local access.
Before getting too far into it, here is the most relevant post in the thread on the FreeBSD-stable list that gives a decent summary of what has changed
Copying/Paraphrasing from that link:
RELENG_9is still alive and will continue for the foreseeable future.
- You can still track 9-STABLE via
RELENG_9_1does not exist (yet?).
- There will be no
RELENG_10*tags on cvs
Rather than being caught off guard for 10.x, switch now, and you might need to anyhow if you plan on tracking anything but 9-STABLE.
A couple links to more FreeBSD+SVN info
Subversion can be installed from
/usr/ports/devel/subversion - and if you plan
on using Apache to serve your SVN repositories over HTTP, you need to check the
option to build the DAV modules when configuring both Subversion and Apache. If
you already have Subversion and/or Apache installed but didn’t select the DAV
module options, you may need to rebuild the ports.
What are the “base”, “doc” and “ports” repositories exactly?
- This includes all of the various FreeBSD OS and related code. Basically if
you were getting
RELENG_<foo>it will come from base. All of the branches and tags are there, they are just worded slightly differently. A few random examples:
# The RELENG_9_0 errata/security branch (9.0-RELEASE-pX) svn co svn://svn.freebsd.org/base/releng/9.0/
# The RELENG_8 stable branch (8-STABLE) svn co svn://svn.freebsd.org/base/stable/8/
# FreeBSD 8.3-RELEASE exactly (no errata) svn co svn://svn.freebsd.org/base/release/8.3.0/
- FreeBSD documentation
- The FreeBSD ports tree. Because the ports tree is not branched the way the
base repository is, you almost always want
svn co svn://svn.freebsd.org/ports/head/
However you can get the ports tree at a certain FreeBSD Release by using a tag such as:
svn co svn://svn.freebsd.org/ports/tags/RELEASE_9_0_0/
Creating a Subversion Mirror
Now, on to the mirroring part. It’s much faster to start your mirror from a seed file, rather than making it completely download and rebuild the entire repository from scratch.
You can fetch from the seed files as described below, and then bring the
repository up-to-date the rest of the way with
svnsync. The doc tree can be
done the same way, but since I don’t build docs I did not mirror it. I used
ftp2.freebsd.org in this example, feel free to substitute your favorite local
mirror, provided it has copies of the files.
# mkdir -p /home/freebsd-svn/ # cd /home/freebsd-svn/ # fetch ftp://ftp2.freebsd.org/pub/FreeBSD/development/subversion/svnmirror-base-r238500.tar.xz # tar xvzf svnmirror-base-r238500.tar.xz # svnsync sync file:////home/freebsd-svn/base # cd /home/freebsd-svn/ # fetch ftp://ftp2.freebsd.org/pub/FreeBSD/development/subversion/svnmirror-ports-r301235.tar.xz # tar xvzf svnmirror-ports-r301235.tar.xz # svnsync sync file:////home/freebsd-svn/ports
Now that you have updated copies of the repositories, you can setup a cron job to run a script to pull in new changes. The script is simple:
#!/bin/sh /usr/local/bin/svnsync sync file:////home/freebsd-svn/base /usr/local/bin/svnsync sync file:////home/freebsd-svn/ports
I added that to a cron job that runs once per hour, but depending on how often you actually update your own servers using it, you could probably dial that back a bit.
Serving Your Mirror
Now that you have a local mirror, but how to serve that up to your other machines? Well, one way is via ssh, and another is svnserve, but for various reasons I prefer to do this via HTTP, so I setup Apache with DAV to hand it out. I’ll also explain svnserve later in this post.
Serving with Apache
First, as I mentioned above, you’ll make sure you compiled both Subversion
/usr/ports/devel/subversion) and Apache (
/usr/ports/www/apache22) with the
relevant DAV module options. Then you can setup the Apache config.
In the main httpd.conf file (
/usr/local/etc/apache22/httpd.conf), check that
the DAV modules are loaded. This may already have been done automatically by the
LoadModule dav_svn_module libexec/apache22/mod_dav_svn.so LoadModule authz_svn_module libexec/apache22/mod_authz_svn.so
Uncomment the line for the
Then save/exit after you’ve made any other changes you want to that file - if you’ve never set Apache up on that server, you’ll probably want to set the server name, admin e-mail, hostname, and point it to a directory that won’t serve up the Apache default files. There are many Apache tutorials out there if you need help with this part.
The first thing I do here is comment out or remove any directives referring to uploading. Then at the end, I add a section that actually allows the SVN bits to be served up:
<Location /freebsd-svn> DAV svn SVNParentPath /home/freebsd-svn Order deny,allow <LimitExcept GET PROPFIND OPTIONS REPORT> Deny from all </LimitExcept> </Location>
Because you can’t exactly commit to these repositories anyhow, that will deny anyone access that tries to write.
After making those changes, save/exit and restart apache. An
configtest would be prudent to ensure the remainder of the config is proper.
Now you should be able to access the repositories over HTTP! Note that you can’t
just browse to
http://your-server/freebsd-svn - you’ll need to use the name of
the SVN repo inside, such as
To checkout the ports tree to your current directory, use:
svn co http://your-server/freebsd-svn/ports/head/ .
Serving with svnserve
If you don’t want to use HTTP, you can setup
svnserve to use the svn protocol
3690. It’s pretty simple actually, just edit
/etc/rc.conf and add:
svnserve_enable="YES" svnserve_flags="-d -R --listen-host 0.0.0.0" svnserve_data="/home/freebsd-svn" svnserve_user="root" svnserve_group="wheel"
-R in the flags tells it to be read only. I would suggest adding an svn user and group to run the daemon under, but it will run as root if you configure it to do so manually as I did in the example.
svnservedoes not appear to properly handle dual stack IPv4/IPv6, so you may need to use the IPv4 IP or a DNS alias without a AAAA when accessing the repo such as:
svn so svn://x.x.x.x/ports/head/ .
If anyone knows a good way to have it listen on both, I’d love to hear it. I’ll update this post if I get that figured out. That’s another reason I prefer HTTP at the moment.
Other Ways to Serve
Accessing over ssh is left as an exercise for the reader. :-)
Here are a couple other handy commands (will add more as I find them or they are suggested):
Overwriting local changes when updating
The default behavior of csup/cvsup was to overwrite your locally changed files. By default Subversion will leave your changes alone and try to merge them. To get the csup style behavior with Subversion, you must revert all local changes first and then update.
svn revert -R /usr/src svn update
csup style behavior is more useful as a consumer applying local patches
and needing to revert to the newest upstream when it’s updated. The svn style
behavior is more useful for developers so they don’t risk losing their work.
This will NOT remove any distfiles or packages from ports, as those have
svn:ignore set in the properties of the ports repository.
Adopting an existing tree
I was hoping to find a way to “adopt” an existing directory into Subversion
rather than deleting and checking out a whole tree again, but so far I haven’t
found a good way to do that cleanly – if anyone knows, I’m all ears. So far
it’s easiest to
rm -rf /usr/src or
/usr/ports when checking out the SVN
copy. I tried using
--force when doing
svn co and it seemed to want to work
but I was left with a giant mess in the end.
Say you want to move from tracking 9-STABLE to tracking 10-STABLE, that’s really easy:
svn sw svn://svn.freebsd.org/base/stable/10
Another practical example, say you start out by grabbing RELENG_9_0 (the security branch):
svn co svn://svn.freebsd.org/base/releng/9.0/
And then the time comes that you want to track 9.1 instead, then you switch like so:
svn sw svn://svn.freebsd.org/base/releng/9.1/
And then you end up with RELENG_9_1, it’s as easy as that. No need to redo the entire checkout.
I always welcome feedback - if you know of a better, easier, or more correct way to do something I’ve described here - I’m more than willing to make corrections.
Much of the information in this post was gleaned from the freebsd-stable thread linked earlier, along with some info from other sources and personal experience.