September 2008

Monthly Archives

MySQL Replication Lag

MySQL Replication Lag

MySQL replication offers a quick and easy solution for having data replicated from a master database to one or more slave databases for failover and/or load balancing of read queries for scalability and is often the first step in a company’s backup and failover strategy.

MySQL replication is completely asynchronous, meaning that the front-end application (often a web site) does not have to wait for replication to happen on each transaction and this keeps the master database operating efficiently.

However, having a completely asynchronous replication solution means that the slave database is never 100% up to date. There is an even more serious issue with MySQL replication because it is single-threaded in nature to ensure transactions are applied to the slave database in the same order that they were applied to the master. This means that if the master database is under load with 100 concurrent transactions for a period of time and the slave is only able to process 1 concurrent transaction due to the single-threaded implementation then the slave database will very quickly fall behind the master database and this could mean thousands or tens of thousands of rows that are not replicated and if the master database fails then these transactions could be lost. I’m sure most companies would find this unacceptable.

I’m curious how developers and DBAs view this issue. It seems unlikely that anyone is entirely comfortable with an unreliable “best effort” replication system for failover although it might be acceptable for scalability if the application is intelligent enough to work around any lag issues optimistically by querying a slave first and then going to the master if the slave is not up to date.

Google have contributed open source code for SemiSyncReplication which provides a partial solution to the problem. Their approach is to have the master sychronously replicate the transaction to the slave mysql instance when the transaction is committed on the master but does not wait for the slave to actually write the transaction to disk, therefore maintaining a sensible balance between performance and reliability.

How do you handle the issue of MySQL replication lag in your organization? Do you use Google SemiSyncReplication or have you found other products/tools to mitigate against replication lag?

FireStorm/DAO 4.0 GA Released

FireStorm/DAO 4.0 GA Released

CodeFutures is pleased to announced general availability for FireStorm/DAO 4.0. The GA version of the product is available for download on the CodeFutures Web site.

The main new feature of this release is support for The Spring Framework. Specifically, FireStorm/DAO now provides an option to generate DAO classes that use Spring JDBC.

There are some significant changes to product packaging with the GA release:

  • The Enterprise Edition is now available completely free of charge and no longer requires a license key. This edition is restricted to generating a maximum of 20 DAO classes per project.
  • The Architect Edition now contains full source code (rather than just the source code for the code generators as in previous releases).
  • The Architect Edition is now sold as a subscription license of $995 per year, which entitles users to technical support and free upgrades (major and minor releases) within the subscription period

About FireStorm/DAO
FireStorm/DAO is a database access tool that adopts a pragmatic approach of generating Java source code for data persistence that is a direct mapping of a particular relational database schema. It is also possible to define complex multi-table queries and to leverage existing database logic contained within stored procedures.

FireStorm/DAO is based on the Data Access Object design pattern and is available in Enterprise, Architect, and OEM editions. FireStorm/DAO Architect Edition allows new custom code generation templates to be developed and integrated with the FireStorm/DAO Studio environment. FireStorm/DAO Architect Edition includes the source code for the Java code generation templates. The code generation templates are written in Java, which means that Java developers have a very short learning curve before they can start customizing the code generation.

Additional information on FireStorm/DAO is available here:

FireStorm/DAO is available for download here:

The Top 10 Things You Should Know About Flex

The Top 10 Things You Should Know About Flex

Alaric Cole, the author of Leaning Flex 3, has produced a list of “The Top 10 Things You Should Know About Flex“:1. Flex is Web Standards, Redefined2. Flex is Flash (and then some)

3. Flex Just Works

4. Flex is Server Agnostic

5. Flex is the Look You Want

6. Flex is Light, and Fast

7. Flex is Accessible

8. Flex is SEO-Compatible

9. Flex is Free, and Open

10. Flex is Easy to Learn

Item 10 is subjective – Flex is easy to get started with but like everything else, experience does count. Item 5 should be promoted to the top of the list – Flex-based applications can often look stunning. Item 9 is true as far as the SDK is concerned. And of course, the list does not mention how Flex fits in with enteprise mashups.


Trend: Single Application Databases

Trend: Single Application Databases

db geek has made a very astute observation about single application databases:One interesting trend that I’ve noticed in many of the organizations that I’ve been into is that increasingly databases are being built to serve single applications. The early visions of databases shared amongst multiple applications is no longer the first choice. To a certain extent this has always been the case for certain operational systems, but now the reach of single application databases has grown. You’ll even find data replicated across multiple multi-terabyte data warehouses to support different business intelligence solutions.

db geek attributes this trend to factors such as commodity hardware and the benefits of reducing the complexities created by multi-application databases.

Database Sharding Lessons

Database Sharding Lessons

Dan Pritchett has written an excellent blog detailing his experience with database sharding while working as an architect at eBay, Inc. and Rearden Commerce:Lesson 1: Right Size Your Shards

Lesson 2: Use Math on Shard Counts

Lesson 3: Carefully Consider the Spread

Lesson 4: Plan for Exceeding Your Shards

Lesson 5: Shard Early and Often

You can read the details of each lesson in Dan’s blog.