Get it from http://developer.apple.com. You will not be able to submit apps to any stores using this XCode version, so turn away if that is something you might want to do.
In XCode's Preferences > Downloads you can install command line tools.
[insert screenshot]
sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer
Visit http://xquartz.macosforge.org/trac/wiki and download and install 2.7.2+.
You will need to fix the symlink it makes:
ln -s /opt/X11 /usr/X11
brew list
Will tell you what you need to check. Try out everything one by one and when one doesn't work brew remove it and then reinstall it. If the install doesn't work, try brew install it --use-gcc to prevent llvm from getting in the way.
Things that gave me issues that I had to remove and install:
- macvim
- imagemagick
- ghostscript