Until we have full provisioning scripts, keep notes about everything that we touch on EC2 in here. ========================= Base DocumentCloud Image ============================ ====================== dcworker has all the steps below ====================== ==================== based on the snap-05235d6c snapshot ====================== * Use the official Canonical Ubuntu 10.04 Lucid Image: ami-714ba518 Use the us-east-1c availability zone. * If bundling an EBS Boot Volume, follow the directions here (mas o menos): http://alestic.com/2010/01/ec2-ebs-boot-ubuntu At step 5, continue with our custom configuration below.... * Switch to a temporary directory. * Update aptitude. * Install these packages: build-essential postgresql postgresql-client postgresql-contrib libpq-dev \ sun-java6-bin git-core mercurial scons libexpat-dev libxml2-dev libxslt-dev \ postfix ruby ri rdoc irb ruby1.8-dev zlib1g-dev libzlib-ruby sqlite3 \ libsqlite3-dev libcurl4-dev checkinstall libbz2-dev graphicsmagick pdftk xpdf \ libitext-java openoffice.org openoffice.org-java-common libtiff4-dev \ libpng12-dev libjpeg62-dev libleptonica-dev tesseract-ocr-dev tesseract-ocr-eng \ xfsprogs libpcre3-dev graphicsmagick * Install RubyGems: DO NOT INSTALL RUBYGEMS THROUGH APT-GET (or you'll live to regret it) wget http://production.cf.rubygems.org/rubygems/rubygems-1.3.7.tgz && \ tar xzvf rubygems-1.3.5.tgz ruby rubygems-1.3.5/setup.rb ln -s /usr/bin/gem1.8 /usr/local/bin/gem sudo gem update --system * Install these gems (--no-ri --no-rdoc): pg sqlite3-ruby rails passenger sinatra right_aws rest-client rack \ bcrypt-ruby rdiscount rubyzip libxml-ruby nokogiri json hpricot calais \ curb daemons cloud-crowd yui-compressor jammit docsplit sunspot sunspot_rails * Add the gem bin directory to the path, in .bashrc. export PATH="$PATH:/var/lib/gems/1.8/bin" * Install Node.js (from source, unfortunately): Nodejs.org (v0.1.99) * Install the SSH credentials (for the current user, root on the EBS chroot): mkdir -p ~/.ssh (or /root/.ssh) Copy the contents of secrets/keys/github.pem to .ssh/id_dsa chmod 0600 .ssh/id_dsa * Checkout DocumentCloud: git clone [email protected]:documentcloud/documentcloud.git ~/documentcloud * Checkout Pixel Ping: git clone [email protected]:documentcloud/pixel-ping.git ~/pixel-ping * Turn off smtpd_use_tsl in /etc/postfix/main.cf. * Check out the latest stable nginx source, then use passenger-install-nginx-module to install a custom nginx (0.8.53), with prefix /usr/local/nginx, and including --with-http_gzip_static_module --with-http_ssl_module * Copy over the nginx.conf config from config/server/nginx to /usr/local/nginx/conf. Copy over the appropriate environment config to sites-enabled. Add /usr/local/nginx/sbin to the PATH in .bashrc. mkdir -p /var/log/nginx * In /etc/sysctl.conf set kernel.shmmax = 852352256 * Copy over config/server/postgresql.conf to /etc/postgresql/8.4/main/postgresql.conf * Copy over config/server/gitconfig.conf to ~/.gitconfig * If bundling, move on to Step 6 of Hammond's directions, but ensure that EC2_CERT and EC2_PRIVATE_KEY are set correctly, and that the size of the EBS is 40 GB. You can use a better name for the $prefix, and a better description. Also, use availability zone 1c. * Clean up the temporary directory. ============================= Postgres EBS =================================== * Follow the directions here: http://deadprogrammersociety.blogspot.com/2009/08/postgresql-on-ubuntu-on-ec2.html Follow Mike's directions to move the Postgres files to the EBS: http://gist.github.com/272258 Copy over configuration from config/server/postgres to /etc/postgresql/8.4/main... pg_ctlcluster 8.4 main start Our postgre DB (volume vol-d4c305bd) is mounted at /data. * Install the HStore module, by cd-ing into the contrib directory of your postgres, and following the directions here: http://www.postgresql.org/docs/8.3/static/contrib.html ============================= Development ==================================== * Install the dependencies and gems listed in PROVISIONING, as appropriate (ie, don't install Tesseract unless you're planning on doing OCR). * Configure Nginx and /etc/hosts to serve dev.dcloud.org locally. An example Nginx configuration is stored in config/server/nginx * Load the development_structure.sql into the database listed in config/database.yml. Load the analytics_structure.sql into the database listed in config/database_analytics.yml * Create your CloudCrowd database, using the config/cloud_crowd/development configuration folder. * Start a local CloudCrowd server and node, with `rake crowd:server:start crowd:node:start` * Start Nginx.