Ruby environment for QA Site

As noted on the previous post, I’m sticking with Ruby 1.8.5 fo QA Sites until I get a convincing argument to upgrade manually — meaning a custom compile. I should really learn how to build RPMs.

Ruby 1.8.5-5.el5_2.3

• official CentOS 5 rpm:
  yum install ruby ruby-devel ruby-irb ruby-ri ruby-libs ruby-docs ruby-rdoc

RubyGems 1.1.1

• manual install:
  wget http://rubyforge.org/frs/download.php/35283/rubygems-1.1.1.tgz
  tar -xvzf rubygems-1.1.1.tgz
  cd rubygems-1.1.1
  ruby setup.rb

Rails 2.0.2

• gem install rails -v=2.0.2

Rake 0.8.1

• installed with Rails

Rspec 1.1.4

• gem install rspec

ZenTest 3.10.0

• gem install ZenTest

test-unit 2.0.2

• gem install test-unit

Capistrano 2.4.3

• gem install capistrano

MySQL 5.0

• database version 5.0.45-7.el5*:
  yum install mysql mysql-devel mysql-server

• ruby mysql drivers version 2.7:
  gem install mysql — –with-mysql-lib=/usr/lib/mysql/ –with-myse=/usr/include/mysql/

PostgreSQL 8.1

• database version 8.1.11-1.el5_1.1:
  yum install postgresql postgresql-devel postgresql-server postgresql-libs
• ruby drivers 0.7.9.2008.03.18:
  gem install pg

  I had trouble installing databases drivers on Windows

  using the older postgres drivers helped, but I think I had to specify the directory:
  gem install postgres

SQLite 3.6.1

• yum install sqlite sqlite-devel (3.3.6-2)
• gem install sqlite3-ruby
• Windows needs the SQLite dll as well as the exe. They need to both be in the PATH environment variable.

# gem list –local

*** LOCAL GEMS ***

actionmailer (2.0.2)
actionpack (2.0.2)
activerecord (2.0.2)
activeresource (2.0.2)
activesupport (2.0.2)
hoe (1.7.0)
mysql (2.7)
pg (0.7.9.2008.03.18)
rails (2.0.2)
rake (0.8.1)
rspec (1.1.4)
rubyforge (1.0.0)
sqlite3-ruby (1.2.2)
test-unit (2.0.0)
ZenTest (3.10.0)

updating (and downgrading) Ruby on QA Site

I’m updating to Ruby 1.8.6 on the Fluffy QA Site.

It’s running CentOS 5 final. (cat /etc/redhat-release)

I added the following yum repository:

[ruby]
name=ruby
baseurl=http://repo.premiumhelp.eu/ruby/
gpgcheck=0
enabled=0

Then I ran:

yum –enablerepo=ruby install ruby ruby-devel rubygems ruby-docs ruby-ri ruby-irb ruby-rdoc

I don’t know the quality of this repository, so don’t take it as my endorsement. I found it on the internet via google which turned up this suggestion on a mailing list.

Most likely the Ruby version supplied by doing an rpm update on the official Centos 5 repository would’ve been fine, Ruby 1.8.5-5.el5_2.3

– note, I’m actually going back to to the official Ruby installation because there is a security vulnerability here:

http://www.securitytracker.com/alerts/2008/Jul/1020476.html

Rather than rip it out by the roots and installing the correct RPMs by force, I found a cool package: yum-allowdowngrade:

yum list available ruby*
Loading “allowdowngrade” plugin
Loading “installonlyn” plugin
Setting up repositories
Reading repository metadata in from local files
Available Packages
ruby.i386 1.8.5-5.el5_2.3 updates
ruby-devel.i386 1.8.5-5.el5_2.3 updates
ruby-docs.i386 1.8.5-5.el5_2.3 updates
ruby-irb.i386 1.8.5-5.el5_2.3 updates
ruby-libs.i386 1.8.5-5.el5_2.3 updates
ruby-mode.i386 1.8.5-5.el5_2.3 updates
ruby-rdoc.i386 1.8.5-5.el5_2.3 updates
ruby-ri.i386 1.8.5-5.el5_2.3 updates
ruby-tcltk.i386 1.8.5-5.el5_2.3 updates

yum grouplist

yum groupremove “Ruby”

yum “groupinstall “Ruby”

yum remove ruby-libs ruby-docs

yum install ruby-libs ruby-docs

yum install ruby-irb ruby-rdoc ruby-ri

so now my ruby installation looks like:

Installed Packages
ruby.i386 1.8.5-5.el5_2.3 installed
ruby-devel.i386 1.8.5-5.el5_2.3 installed
ruby-docs.i386 1.8.5-5.el5_2.3 installed
ruby-irb.i386 1.8.5-5.el5_2.3 installed
ruby-libs.i386 1.8.5-5.el5_2.3 installed
ruby-mode.i386 1.8.5-5.el5_2.3 installed
ruby-rdoc.i386 1.8.5-5.el5_2.3 installed
ruby-ri.i386 1.8.5-5.el5_2.3 installed

The reason I wanted to upgrade to 1.8.6 was to get the 1.9 warnings, and to keep consistent with windows. I’ll still use 1.86 on Linux VMs not facing the internet, but not on QA Sites.

rubygems 1.1.1 was installed by hand.

PHP Continuous Integration

Xinc is a PHP continuous Integration server

Not sure which site is current. My best guess is google.

http://sourceforge.net/projects/xinc

http://code.google.com/p/xinc/

Some articles:

http://php5.phpmagazine.net/2008/05/introducing_to_xinc_continuous.html

http://www.slideshare.net/arnoschn/continuous-integration-and-php

Also look at CruiseControl + phpUnderControl

PHPUnit (or SimpleTest?)

Phing (build tool) or ant (Cruisecontrol can also use Rake, what about Capistrano?)

SVN, CVS, Perforce, etc

Continuous integration tools

Here’s a link to a large matrix of CI tools:

http://confluence.public.thoughtworks.org/display/CC/CI+Feature+Matrix

Here’s my own list with basic impressions

• CruiseControl – a bit old
• Continuum – Apache
• Luntbuild – Dead simple, but not that intuitive or extendible.  Usually end up cloning and tweak
• Hudson – Looks very interst
• Anthill – not open source?
• Bamboo – Atlassian
• Teamcity – Intellij
• BuildBot – Python based
• Cerberus – Ruby based
• CruiseControl.NET – .NET based, I hear it’s much more featureful than regular cruisecontrol.

New mailserver: exim + dovecot

I set up exim + dovecot for qa-site. I’m pretty happy with the results so far. Installation was a snap, and configuration was practically pain free (thanks in part to google and this mini howto for SMTP auth.)

I created SSL keys for both exim and dovecot (that are practically identical) using this command:

openssl req -x509 -newkey rsa:1024 -keyout exim.key -out exim.crt -days 365 -nodes

openssl req -x509 -newkey rsa:1024 -keyout dovecot.key -out dovecot.crt -days 365 -nodes

and then edited the configuration files to point to them:

exim.conf:

tls_certificate = /usr/share/ssl/certs/exim.crt
tls_privatekey = /usr/share/ssl/private/exim.key

dovecot.conf:

ssl_cert_file = /usr/share/ssl/certs/dovecot.crt
ssl_key_file = /usr/share/ssl/private/dovecot.key

in exim.conf, I also made the following changes:

primary_hostname = qa-site.com #this would have used `uname -n` by default

uncommented:

auth_advertise_hosts = ${if eq {$tls_cipher}{}{}{*}}

and commented:

#auth_advertise_hosts =

and then added:

plain:
driver = plaintext
public_name = PLAIN
server_prompts = :
server_set_id = $2
server_condition = “${if pam{$2:$3}{1}{0}}”

login:
driver = plaintext
public_name = LOGIN
server_prompts = “Username:: : Password::”
server_set_id = $1
server_condition = “${if pam{$1:$2}{1}{0}}”

in addition, to use maidir, I edited exim.conf to include:

local_delivery:
driver = appendfile
# file = /var/mail/$local_part
delivery_date_add
envelope_to_add
return_path_add
group = mail
mode = 0660
maildir_format = true
directory = /home/${local_part}/Maildir
create_directory = true
check_string = “”
escape_string = “”

in dovecot, I just had to set:

mail_location = maildir:~/Maildir

I created a .muttrc file to read my Maildir:

set mbox_type=Maildir
set folder=”~/Maildir”
set mask=”!^\\.[^.]“
set mbox=”~/Maildir”
set record=”+.Sent”
set postponed=”+.Drafts”
set spoolfile=”~/Maildir”

I now have working TLS with SMTP auth, POP3 and IMAP access through remote thundirbird/outlook and local mutt clients. They both use PAM system-auth by default (exim needed read access to /etc/shadow.)

The next steps are to set up procmail to use spamassassin and clamav to filter incoming mail, and set up a webmail client. I mentioned on my fijiaaron blog that I’m looking for webmail application recommendations. I’m leaning towards v-webmail.

One last thing will be to set up virtual domains, so that, for example, cuencatravel.qa-site.com can have it’s own email setup.

My Linode was rebooted

I don’t know what happened, but httpd and mysqld were stopped. I don’t want to babysit servers. I hope this isn’t going to be a problem with Linode.

# uptime
22:06:13 up 2 days, 16:36,  1 user,  load average: 0.01, 0.02, 0.00

SSL logins

Next on the list is to setup Bugzilla, Trac, Subversion logins to only use SSL.

Also for wiki, etc.

Trac setup

Make sure apache, python, mod_python, subversion, sqlite, clearsilver, & clearsilver-python are installed and working first.

Trac uses the swig bindings installed with Subversion, not pysvn.

– install setuptools for genshi and pygments (needed for 0.11, not for 0.10)

# wget http://peak.telecommunity.com/dist/ez_setup.py
# python ez_setup.py
# easy_install genshi
# easy_install pygments

# easy_install docutis

– optionally install silvercity for syntax highlighting

# wget http://ufpr.dl.sourceforge.net/sourceforge/silvercity/SilverCity-0.9.7.tar.gz
# tar -xvzf SilverCity-0.9.7.tar.gz
# python setup.py install

– Download, unzip, install trac:

# wget ftp://ftp.edgewall.com/pub/trac/trac-0.10.4.tar.gz
# tar -xvzf trac-0.10.4.tar.gz
# python ./setup.py install

— Create a project:

# trac-admin /var/www/trac-site initenv

– edit trac.ini (no changes yet)

– test with build in server

# tracd –port 8000 /var/www/trac-site

– configure apache to point to trac site
<Location /trac>
SetHandler mod_python
PythonInterpreter main_interpreter
PythonHandler trac.web.modpython_frontend
PythonOption TracEnv /var/www//trac-site
PythonOption TracUriRoot /trac
</Location>

The trac documentation is actually pretty good and this closely follows it.

http://trac.edgewall.org/wiki/0.10/TracInstall
http://trac.edgewall.org/wiki/TracModPython

I did have some trouble with getting python and subversion working together on windows, however (that may have had to do with python-2.4, I don’t think I tried downgrading to 2.3, and It might have had trouble with newer Apache 2.2.4)

And fast-cgi isn’t an option on windows.

Subversion

Apart from being used as a developer repository, subversion is used by Trac.

Subversion installation was pretty straightforward.  I installed the RPMs with yum:

# yum install subversion mod_dav_svn

–Optionally install bindings for languages

# yum install subversion-perl subversion-ruby  subversion-java

– create a repository

# svnadmin create /var/www/svn/repository-name

– edit httpd.conf (the RPM creates /etc/httpd/conf.d/subversion.conf)

LoadModule dav_svn_module     modules/mod_dav_svn.so
LoadModule authz_svn_module   modules/mod_authz_svn.so
– create a location

<Location /svn>
Options Indexes
Order allow,deny
Allow from all

DAV svn
SVNParentPath /var/www/svn

<LimitExcept GET PROPFIND OPTIONS REPORT>
AuthType Basic
AuthName “Subversion Login”
AuthUserFile /var/www/access/.htpasswd
Require valid-user
</LimitExcept>
</Location>

– restart apache

Note for all you rocketscientists who use the SVNParentPath directive:

http://localhost/svn returns 404 Forbidded.  Navigate to http://localhost/svn/respository-name  works.

Set up svn to use HTTPS optionally.

Bugzilla dependencies

I installed bugzilla after fixing the ImageMagick stuff this morning. This wasn’t quite straightforward either, but I’d waded through this before with one-shore and a local install on windows and a CentOS VMWare image.

Here are my notes:

– download bugziila 3.0.3
# wget http://ftp.mozilla.org/pub/mozilla.org/webtools/bugzilla-3.0.3.tar.gz

– unzip bugzilla
# tar -xvzf bugzilla-3.0.3.tar.gz

– move bugzilla to site
# mv bugzilla-3.0.3 /var/www/bugzilla

– restart websever
# service httpd configtest
# service httpd restart

– check bugzilla dependencies
# cd /var/www/bugzilla
# perl checksetup.pl

– install dependencies via CPAN
cpan Bundle::CPAN
cpan Bundle::Bugzilla

– this failed on the DBD::mysql 4.0 plugin (3.0 should have workd for Bugzilla 3.0.3, though 4.0 is required for Bugzilla 3.1.x)

I fixed that with a manual install of the CPAN module and temporarily setting the root password to “” to match what was expected in the test script. Alternately, you can specify the default username & password to test with (somewhere).

After this, the basics were installed, but I got the extras with:

# cpan Email::Send
# cpan Email::MIME:Modifier
# cpan Template::Plugin::GD
# cpan SOAP::Lite
# cpan HTML::Scrubber
# cpan Email::MIME::Attachment::Stripper
# cpan Email::Reply

I had to manually install chart:

– manually install Chart
# wget http://search.cpan.org/CPAN/authors/id/C/CH/CHARTGRP/Chart-2.4.1.tar.gz
# tar -xvzf Chart-2.4.1.tar.gz
# cd Chart-2.4.1
# perl Makefile.PL
# make && make test && make install

– create bugzilla account on mysql
#mysql -u root -p
mysql>grant all on bugzilla.* to ‘bugzilla’@'localhost’ identified by ‘********’;

– edit bugzilla localconfig
# vi ./localconfig
$db_driver = ‘mysql’;
$db_host = ‘localhost’;
$db_name = ‘bugzilla’;
$db_user = ‘bugzilla’;
$db_pass = ‘********’;

– install Bugzilla by running checksetup.pl again
# perl checksetup.pl

After this, it was just some playing with Apache to get rid of the 404 Forbidden (I’ll probably forget what I did again) and then configure via the admin login.  I had a brief problem with cookies because I tried to set the cookie domain — that will take some more work with mulitple installs, but apart from the qa-site.com and demo.qa-site.com being hosted together, it won’t be a problem with VPSes.