[nflug] Part Time Contract Opportunity

[Erek Dyskant]

I read your email again and realized I didn't really address each need
individually.


>         1) TOP PRIORITY: Database backup/mirror 
>         I'd like to install MySQL on one of the web servers (they're
>         under-utilized) and use it as a mirror of the live database to
>         ensure we have a reliable backup.
>         
>         I'm much more concerned that it serve as a fool-proof recovery
>         solution than a fall-over in the event of a db problem.
	Definitely possible.  I mentioned strategies in the previous email.
>         
>         
>         2) SECONDARY PRIORITY: Database server tuning
>         We're pretty much running on the stock-box set-up provided by
>         SoftLayer. I would imagine that there would like be some
>         degree of system tuning or settings modifications that might
>         greatly enhance the overall performance of the machine.
	I could have a look at this, and certainly some tuning can be done with
Apache and with MySQL.  At the system level there's definitely some
tuning, but my experience is that when you're in the 3-4 server range,
the default system options are pretty good.
	Another place we could look at to score some performance improvements
is optimizing the caching and the MySQL queries.  Depending upon the
exact application you're looking at, you may be able to cache things
rather than look them up every time.
>         
>         3) THIRD PRIORITY: Database modifications
>         Back in the day, when things were much slower, we added a
>         full-text index to the table used to contain postings. This
>         was great for full-text Boolean searches, but now such
>         searches take several seconds against the 3GB full-text
>         index... so the service is now disabled. I would imagine that
>         maintaining the full-text index introduces an unneeded degree
>         of server load each time a new post is added... can the
>         full-text index be safely disassociated in the database?

I'll give this one away from free:

SHOW INDEX FROM {TableName};

Find the key name that corresponds to your fulltext index.

DROP INDEX {key_name} ON {TableName};

Disclaimer:  This should be tested in a development environment first.
I have not tested it and it's based upon my recollection of the MySQL
syntax.

	If you want full-text queries back, you may look at a dedicated search
indexing box: Either a Lucene machine or a Google Search Appliance.  The
Google appliances are more plug-and-play, but Lucene
gives you finer control over what metadata gets searched.

>         
>         4) FOURTH PRIORITY: Contract Support
>         We need someone reliable to be our contract support and check
>         in on the boxes occasionally and ensure they're properly
>         hardened and all is running well... in addition to responding
>         to any issues that might arise. 
	Of course.  You also should have an automated issue tracking system in
place if you don't already.


>         Other considerations... the hardware list I gave you was
>         deployed 7 months ago and we've pretty much had to upgrade our
>         hardware every 9-12 months over the past four years (due to
>         growth). Our traffic is currently 2x what it was when the
>         hardware was deployed, so we'd need your best-guess on how
>         high is up with these machines so we can consider how and when
>         to upgrade again.
	Probably the best way to do this would be to run some benchmarks during 
a non-peak period of usage and find out how high you can go before the response times
become unacceptable or you start dropping connections.

	Ultimately I think that you're going in a direction where you need
to look at more scalable systems.  It may be a buzzword but it applies.  So, a 
web server cluster that reads against a SAN and a documented system for deploying 
new servers, and a scalable database solution.  For the database you're looking
at either a MySQL cluster or a medium-iron system (4 or 8 sockets, fast storage, etc.)


Hope this helps and gives you more thought into the sorts of ways I
approach these problems.


Regards,
Erek Dyskant