Hosting a git repository on dreamhost
[This entry updated April 14, 2008 based on comments and this blog. Updated again August 26, 2008.]
I use dreamhost.com as my hosting provider and wanted to use it to host git repositories as well. git supports publishing via HTTP so that part is pretty easy, but I also wanted to make it easy to push my local work to the repository on dreamhost.
Unfortunately, git documentation was a little impenetrable the first few times I read it and dreamhost doesn’t offer git automatically, so here is a summary of what I did in case it helps
someone else figure it out.
Throughout, I use “USER” and “MACHINE” instead of my username and shell machine at dreamhost and “example.com” instead of my actual web address. Be sure to replace these with your own information, as appropriate.
Step 1. Compile git on dreamhost
Following various instructions on the web, I downloaded and compiled git in my dreamhost shell account. The NO_CURL option is used because that library isn’t available and I only need git on dreamhost to receive pushes, not to pull from other repositories. The NO_MMAP option is used to keep dreamhost from killing git for appearing to take too much memory.
This sample installs git in standard directories under the user’s home directory (e.g. /home/USER/bin). That may or may not be what you want. You will need $HOME/bin (or whatever you choose) in your PATH.
$ mkdir ~/src $ cd ~/src $ wget http://www.kernel.org/pub/software/scm/git/git-1.5.4.rc4.tar.gz $ tar xzf git-1.5.4.rc4.tar.gz $ cd git-1.5.4.rc4 $ ./configure --prefix=/home/USER/ NO_CURL=1 NO_MMAP=1 $ make $ make install $ git --version
Step 2. Creating a repository
I created a ‘git’ subdirectory under my web root to hold my repositories.
$ mkdir ~/example.com/git
Creating a new, empty project repository required these steps, which I later automated as a bash function. (Stick it in your .bashrc)
newgit()
{
if [ -z $1 ]; then
echo "usage: $FUNCNAME project-name.git"
else
gitdir="/home/USER/example.com/git/$1"
mkdir $gitdir
pushd $gitdir
git --bare init
git --bare update-server-info
chmod a+x hooks/post-update
touch git-daemon-export-ok
popd
fi
}
Then creating a new repository is as easy logging into dreamhost and running newgit repo-name.git. Or, I can even do it remotely via ssh:
$ ssh USER@MACHINE.dreamhost.com newgit repo-name.git
Step 3a. Initializing a repository (from scratch)
Before the repository can be used, it needs to be initialized with some data. Assuming that this is a new project, I have to create a local repository and publish it to the dreamhost repository.
From my development machine, I can create an empty git repository:
$ mkdir repo-name.git $ cd repo-name.git $ git init
Next I need to create some files to upload — in this case just an empty ‘.gitignore’ file:
$ touch .gitignore $ git add .gitignore $ git commit -m "just gitignore"
Since this repository was created locally instead of with ‘git clone’, I have to manually setup the dreamhost repository as the ‘origin’ source. I use ssh so I will be able to write to it:
$ git remote add origin \ ssh://USER@MACHINE.dreamhost.com/home/USER/example.com/git/repo-name.git $ git config branch.master.remote origin $ git config branch.master.merge refs/heads/master
Finally, I can push all the local repo contents to the dreamhost repository and then pull it back to make sure everything starts in sync:
$ git push --all $ git pull
Step 3b. Initializing a repository (from an existing repository)
If I want to initialize the repository from an existing repository somewhere else, I can clone the existing repository to my development machine and then push it to dreamhost. In this case, I use the ‘-o’ flag to set the name of the clone to ’source’:
$ git clone -o source http://existing.source.example.com/git/repo-name.git $ cd repo-name.git
Then, just as in (3a) above, I set up dreamhost as my ‘origin’ (the default for pull/push):
$ git remote add origin \ ssh://USER@MACHINE.dreamhost.com/home/USER/example.com/git/repo-name.git $ git config branch.master.remote origin $ git config branch.master.merge refs/heads/master $ git push --all $ git pull
Now I can update my repository from the original source, make changes, and then publish back to my dreamhost repository:
$ git pull source master hack...hack...hack... and commit $ git push
Step 4. Pulling from and pushing to the repository
Once the repository is initialized with data, people can clone the repository in a couple ways. One option is via http:
$ git clone http://example.com/git/
Or, if I want to clone the repository — perhaps on a different development machine, I can do that with ssh if I want to be able to push changes back:
$ git clone \ ssh://USER@MACHINE.dreamhost.com/home/USER/example.com/git/repo-name.git
Because I used “clone” with an already-initialized repository, the ‘origin’ information is already set correctly and I don’t have to follow the steps laid out in (3a) or (3b). I can just pull or push back directly to the ‘origin’ location:
$ git pull $ git push
Step 4. Setup gitweb
git comes with gitweb to provide a nicer web browser interface. In my case, I put it right into the directory that holds my .git repositories. (I deleted an extra file I didn’t need and set permissions to be suexec friendly.)
$ cd ~/example.com/git $ cp ~/src/git-1.5.4.rc4/gitweb/git* . $ rm gitweb.perl $ chmod 755 gitweb.cgi
I also created a runtime configuration file gitweb_config.perl like this:
# where's the git binary?
$GIT = "/home/USER/bin/git";
# where's our projects?
$projectroot = "/home/USER/example.com/git";
# what do we call our projects in the ui?
$home_link_str = "USER projects";
# where are the files we need for web display?
@stylesheets = ("gitweb.css");
$logo = "git-logo.png";
$favicon = "git-favicon.png";
# what do we call this site
$site_name = "USER git repositories";
This kind of configuration can also be done at configure/make time, but having a file makes it easier to tweak and customize later.
I created a .htaccess file that enables CGI and redirects requests for the index to the CGI file:
Options +ExecCGI RewriteEngine On RewriteRule ^$ gitweb.cgi RewriteRule ^([?].*)$ gitweb.cgi$1
Now browsing example.com/git gives a nice interface to any repository under that directory.
Other resources:
- Git User Manual - Sharing development with others
- Dreamhost Wiki - Git (see Setup Three)
- How_I_set_up_gitweb (ianloic.com)
- Publishing Git Repositories (madduck)
- Hosting Git Repositories on Dreamhost (tail -f development.log)
March 12th, 2008 at 10:42 am
Hey, Nice this tutorial!
Actually I think THIS should be on dreamhost wiki! =)
Regards,
March 22nd, 2008 at 6:11 pm
[...] In my case, I was interested in setting up our lab server (OSX 10.3) to properly host git repositories. First, install git on the server. Next, add the following to /etc/profile, which is a handy script to make repository creation easy. The script is great and is from here. [...]
March 30th, 2008 at 7:38 am
[...] Hosting a git repository on dreamhost [...]
April 5th, 2008 at 3:41 am
Excellent!
Thanks for the info!
April 9th, 2008 at 10:09 am
[...] Most of this is taken from this post. [...]
April 9th, 2008 at 8:30 pm
just wanted to drop by and say thanks! saved me a lot of time :)
April 13th, 2008 at 3:29 pm
I’m getting the following error when I try to clone from my DH account:
cpjolicoeur@~ $ git clone ssh://cpjolicoeur@wiggum.dreamhost.com:/home/cpjolicoeur/git.craigjolicoeur.com/gspolitics.git
Initialized empty Git repository in /Users/cpjolicoeur/gspolitics/.git/
Bad port ”
fatal: The remote end hung up unexpectedly
fetch-pack from ’ssh://cpjolicoeur@wiggum.dreamhost.com:/home/cpjolicoeur/git.craigjolicoeur.com/gspolitics.git’ failed.
any idea why the fetch-pack command is failing? is there somekind of git-daemon that needs to be running on my DH box that isnt?
April 13th, 2008 at 4:06 pm
Dr J — you’ve got an extra colon after wiggum.dreamhost.com. I think that’s the problem.
April 13th, 2008 at 5:20 pm
I had that colon because I thought that was standard SSH path layout. So I removed the colon and now I get this error:
cpjolicoeur@~ $ git clone ssh://cpjolicoeur@wiggum.dreamhost.com/home/cpjolicoeur/git.craigjolicoeur.com/gspolitics.git
Initialized empty Git repository in /Users/cpjolicoeur/gspolitics/.git/
fatal: no matching remote head
fetch-pack from ’ssh://cpjolicoeur@wiggum.dreamhost.com/home/cpjolicoeur/git.craigjolicoeur.com/gspolitics.git’ failed.
different error “no matching remote head” instead of of “the remote hung up unexpectedly.”
still not working though and I can’t seem to find anything on the interwebs about why
April 13th, 2008 at 6:06 pm
nm. i found the solution. the problem is that the clone was failing because the remote repository was bare.
:
April 14th, 2008 at 9:12 am
[...] chose to simply compile Git locally into my DH account and connect to it over SSH. This blog post on Autopragmatic.com got me about 90% of the way toward completion, so the next few steps are taken directly from his [...]
April 16th, 2008 at 11:45 am
Thanks for the helpful post. One concern I have about this setup is having the repository itself stored in a web-accessible directory. If all access will be done via ssh, it makes more sense to have a ~/git directory to hold the actual codebase, and then on git.example.com only have gitweb which can point to the repositories and allow online browsing.
April 16th, 2008 at 5:19 pm
ssh is not only way to way to access the data. It’s also possible for git to use direct http access as well, which is why it’s in a web-accessible directory — e.g. “git clone http://example.com/git/project-name.git“.
That form is useful for providing read-only access to people that you’re collaborating with. Note that specified that way, it doesn’t go through gitweb.
April 17th, 2008 at 12:25 pm
While this is true, I just have general qualms about placing my source repositories in a web-accessible directory. I wish there was an Apache module mod_dav_git that works like mod_dav_svn!
May 17th, 2008 at 2:48 am
Thanks, great write up!
Couple of gotchas (okay they’re a bit obvious but…):
- add $HOME/bin to your path (if it isn’t already)
- watch for curly quotes and n-dashes when copying the newgit function
Also, the latest git version can be installed:
http://www.kernel.org/pub/software/scm/git/git-1.5.5.1.tar.gz
May 17th, 2008 at 7:13 am
Fixed the curly quotes and n-dashes — turned off WordPress ‘texturize’ filter. I didn’t realize it did that. Very annoying.
Thanks for the other tips.
June 10th, 2008 at 6:57 am
[...] Due credits goes to Autopragmatic. [...]
June 20th, 2008 at 3:33 am
What do these two lines do?
$ git config branch.master.remote origin
$ git config branch.master.merge refs/heads/master
Great article! Thanks for all the help!
June 20th, 2008 at 6:17 am
Those lines define the default remote repository and branch to use with “git pull” and “git push”. When you “git clone …”, those get set automatically exactly like this so that the clone source is the default. In this case, I always want my repository to be the default.
June 29th, 2008 at 3:19 am
[...] over WebDAV. Determined to continue, I instead tried the (somewhat more complex) steps at Autopragmatic » Blog Archive » Hosting a git repository on dreamhost. Even though this involved rebuilding a slightly non-standard version of Git on the server this now [...]
July 22nd, 2008 at 4:16 pm
You forget say to make chmod 755 in the repo dir. If not, dreamhost not allow you to run the gitweb.cgi.
August 4th, 2008 at 9:22 pm
I get
fatal: I don’t handle protocol ‘ ssh’
when trying to push for the first time. Any ideas? Thanks!
August 5th, 2008 at 10:34 am
Perfect, thanks.
August 5th, 2008 at 10:51 am
Jim — that sounds like an issue with the git client. If you’re trying with the version you compiled as per the instructions, it excludes the CURL library, so some of the normal client functionality isn’t available. These instructions just get the server working in a minimal way. You’ll need a fully-functional client elsewhere to be able to push via ssh.
August 8th, 2008 at 10:57 am
Got this error:
bash: git-receive-pack: command not found
fatal: The remote end hung up unexpectedly
After a bit of searching, I found this at
http://www.bluestatic.org/blog/2007/08/01/git-public-push-ing/ :
So I’ve been using git locally for a few days now, and I decided it would be a good idea to try push-ing it to bluestatic.org to make sure that public repositories will actually work! After many, many failed attempts all I could get was this cryptic message:
bash: git-receive-pack: command not found
fatal: The remote end hung up unexpectedly
error: failed to push to [removed]
And then, by chance, and after a good 20 minutes of searching, I found this perfect answer on Google:
Many installations of sshd do not invoke your shell as the login shell when you directly run programs; what this means is that if your login shell is bash, only .bashrc is read and not .bash_profile. As a workaround, make sure .bashrc sets up $PATH so that you can run git-receive-pack program.
So I solved this error by simply creating a .bashrc file with this in it:
export PATH=${PATH}:~/bin
I thought I’d just write about this in hopes of helping others out.
August 26th, 2008 at 12:39 am
Thanks for the tutorial! Good stuff, however line 5 of the first block of shell instructions wasn’t correct for me (when copying/pasting). You have:
cd git-1.5.4.rc.tar.gz
but the archive decompresses to “git-1.5.4.rc4″ as the directory name, meaning the command should be:
cd git-1.5.4.rc4
Thanks so much :)
August 26th, 2008 at 6:46 am
Thanks. I’ll fix it.
August 26th, 2008 at 11:45 pm
Thanks for this guide - and for keeping it updated. I just noticed that git was already installed on my Dreamhost server. It’s version 1.5.4.1. I tried your instructions without building git myself… and it worked fine. So people may want to check if git is already installed.
October 15th, 2008 at 7:51 am
Hello. I completed my installation following instructions from here and various other places on the net. Rather than putting my repository in http://domain.com/git I decided to use a subdomain, http://git.domain.com. This works fine when cloning and push and pulling using the FULL address i.e.
git clone ssh://me@server.dreamhost.com/home/me/git.domain.com/. However, I cannot access the subdomain directly from ssh. Example, the following fail,
git clone ssh://me@git.domain.com/.
Which is a shame because that the reason I wanted to use a subdomain! Do you have any ideas on what I need to do? For example, I have created this subdomain in the Dreamhost control panels but I have not setup any mirrors of forwarding? Perhaps I need to do this?
Regards,
November 16th, 2008 at 12:43 pm
This guide is excellent. I’ve used it a few times now—Thanks a lot!