How to skyrocket MySQL performance the easy way


When you see your average page loading time going from 0.08 sec to 1.40 sec you know it’s time to do someting about it. I’ve run into this situation a few days ago and made some tweaks to my database. Now the average loading time is even lower at 0.05 sec. Here is how to do it.

MySQL logo

1. Create indexes

Indexes are a way for mySQL to scan through tables very fast. The greater the number of rows, the greater the speed improvement. The mySQL manual claims that in a table with 1000 rows, an index could speed up data retrieval by 100 times.

Let’s see how it works in action. Let’s assume for example that we have a simple application that stores all books in a public library. We will need a books table and an authors table to store our data. Since an author may have written many books, and some books can have multiple authors (like many tech books) this is a many to many relationship. So we also need a books_authors table for relating books to authors. So our tables will be:

1. books

CREATE TABLE books (
  ID INT,
  title VARCHAR(255),
  ISBN VARCHAR(255),
  date_published (date)
)

2. books_authors

CREATE TABLE books_authors (
  ID INT,
  book_id INT,
  author_id INT
)

3. authors

CREATE TABLE authors (
  ID INT,
  name VARCHAR(255),
)

Primary keys are automatically indexed by mySQL so we will turn our attention to the books_authors table. The reason is that there will be a lot of work done by mySQL in that table, since it acts as the glue for the other two. Depending on the type of your select queries you may want to index other columns as well, such as the ISBN column if you want to search for books by their ISBN. However if you make the ISBN unique it will be also automatically indexed by mySQL. This is a great idea since ISBNs are unique anyway.

However, back to our example. Let’s see the code that will create the indexes.

CREATE INDEX book_id_index ON books_authors (book_id);
CREATE INDEX author_id ON books_authors (author_id);

As you may have guessed, the syntax goes like this.

CREATE INDEX index_name ON table (column);

 

2. Query cache

MySQL has the feature of caching select queries along with their results. To make things even better, MySQL serves the same cached results to different clients. A great advantage over classic server side HTML caching. Although query cache is on by default you have to configure the cache size in order for it to work.

To check out the size of the query cache connect to MySQL and issue the following command:

SHOW VARIABLES LIKE 'query_cache_size';

The size of query cache should depend on your server’s RAM. A nice practice is setting it to about half the memory size to allow resources for other services.

To set query cache size to 1MB just do:

SET GLOBAL query_cache_size = 1000000;

The first time a select query is executed it will run at normal speed. However consecutive queries will be returned almost instantly since they will be cached. This will speed up even the most simple applications since routine queries, like “select the 10 latest posts” will practically always be cached.

 

Conclusion:

These two simple tips can have an enormous impact on your web application’s performance. However you should be aware that database optimization is a huge topic and it’s worth digging deeper into it. Your next stop from here should be database normalization.

 

Reference:

How MySQL Uses Indexes
http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html

When To Use Indexes In MySQL
http://www.howtoforge.com/when-to-use-indexes-in-mysql-databases

Query Cache Configuration
http://dev.mysql.com/doc/refman/5.1/en/query-cache-configuration.html

If you got comments/suggestions or want updates on new posts be sure to follow me on twitter.

The business renaissance and the culture of open source


In the last couple of days I’ve had the “problem” of having lots of people trying out deskhot. The nice part of my “problem” is that people, not only love the application and start using it right away, but they also move their bookmarks from competitor’s services. The bad part is that the database has gotten very large in a very short period of time.

The creation of adam

I’ve spent the last two days conducting a series of experiments and benchmarks. I’m thinking of replacing apache with nginx and MySQL with a no-db solution like MongoDB. I’ve set up Debian Linux on a couple of computers, installed the software mentioned above and have been testing various configuration combinations since. While I was feeling lost and dizzy from hours in front of a green and black terminal, countless manual pages and forum posts, I remembered how computing was back in the 90’s when I was studying computer science.

Back then it was all about things like Windows NT, enterprise UNIX, Visual Studio and the like. If you wanted to even have a shot at creating a software business you needed money. And by money I mean lots of money. But the real cost was not hardware or software. It was the time and money you had to invest in specialized training from corporations like Microsoft and SUN. Back in 1998 you couldn’t just find a specialized technical answer on Google.

In about a decade the open source movement changed all that. Now everyone who is willing to devote a reasonable amount of time to it, can start a software business. The same enterprise level software available to Google and Facebook is available for free to everyone. Along with documentation and technical support. It’s the equivalent of every school basketball team on earth being able to be coached by Phil Jackson for free.

It’s becoming obvious that over the next twenty years we will experience radical changes in the way businesses are created and operated. The Goliaths of today will be smaller tomorrow and the Davids of today will have a better chance at the future. And the best part of this forthcoming renaissance is that we can all be a part of it.

Einstein would have used Linux


If you’ve visited any tech blog over the last couple of years you must be convinced by now, that Apple is creating amazing products that enhance our creativity and productivity. And why is that ? Not only they are technologically superior but they got a certain different culture. An aura surrounding any Apple product. It’s the same culture embraced by the alternative and successful people. The artists, the rebels, the warriors, the crazy ones.

It sure sounds amazing. You get one of those Apple computers and you join the elite of the mankind. However did you notice something weird about this video? None of these people ever used a mac. In fact most of them lived before the era of personal computing. But what if they had been given the opportunity of having a mac ? Well that’s an easy guess!

Einstein would be too smart for it and would have installed Linux. Muhammad Ali would have repeatedly punched the monitor right away. Bob Dylan and John Lennon would probably be too busy scratching their guitars. Sorry if I spoiled the recipe for success but it really is not that easy.

As a web developer I have many colleagues who praise and love their macs. I don’t think that’s the right mentality for any professional. The moment you start caring about the kind of tools you use is the exact same moment when you give away your focus and passion to Apple’s marketing division for free. It’s not about the tools you fools, it’s about what you do with them.

It’s the right time to quote Hugh MacLeod who wrote in his “How to be creative” manifest:

A fancy tool just gives the second-rater one more pillar to hide behind.
Which is why there are so many second-rate art directors with state-of-the-art Macintosh
computers.
Which is why there are so many hack writers with state-of-the-art laptops.
Which is why there are so many crappy photographers with state-of-the-art digital cameras.
Which is why there are so many unremarkable painters with expensive studios in trendy
neighborhoods.
Hiding behind pillars, all of them.

Selling people ? Well that pisses me off


Thousands of people use delicious. Until a few months ago I was an active daily user myself. But delicious is slow and I always thought it’s social layer was implemented poorly. I never felt I was interacting with people of similar interests on delicious. Yahoo never had a vision for delicious. They never cared to improve it so after a while I got tired.

I’m a web developer, and web bookmarking is essential to me. I needed a fast and easy way to store my bookmarks. And a personalized homepage so I can get faster to them. I thought “if nobody cares about the tools I need, I should create them myself”. So I created Deskhot to cover my own needs at first.

I showed it to a few friends and they started using it. It turned out I wasn’t the only one who needed a fast bookmarking tool. Now Deskhot is used by a small group of people who moved away from delicious and never looked back.

A few days ago Yahoo decided to finally sell delicious along with it’s users. They warned that everyone not willing to accept the new terms of service will not be able to use delicious in a few months. What they really said is: I don’t care if you got 5 years of bookmarks in there. You will take parts in some investor’s “social web” experiments like it or not.

If you, like me and lots of others, don’t like being sold, but just need a great tool for your bookmarking, you are welcome to join us. I have created an import tool that allows you to migrate your bookmarks from delicious in a few minutes along with their tags and privacy settings. Most pages in Deskhot load as fast as 0.08 seconds and real time search makes finding your bookmarks faster than ever.

Deskhot will always be free of charge. I will be offering premium accounts for people who desire specialized features like storing cached pages. But the main functionality of Deskhot will always be free of charge. By having premium accounts along with the free ones I will ensure that Deskhot will always be a profitable project that will never have a reason to be sold or shut down.

I’d like to hear your opinion about it. Feel free to talk to me on twitter anytime.

April roundup


April has been a great month with new stuff and improvements for the existing ones. More new people started using Deskhot and the number of stored bookmarks is about to hit the first milestone. I have installed analytics on Deskhot in an effort to get more information on the way people use it and make future improvements based on real-world usage.  Besides these and lots of bug fixes for Deskhot here’s what new came out of the lab over the last month.

Import bookmarks on Deskhot

Deskhot can now import your bookmarks from your browser or delicious. This was one of the features planned for later but certain events made this top priority. An extensive post will go online in a few days with more details on this.

deskhot import bookmarks

 

New favicon for Deskhot

When you have 10 or more tabs open and you need to get to your bookmarks right away, till now you had to scan the tab bar titles for deskhot. This was taking some time and was a bit annoying. I created a new favicon in plain blue and white. The new favicon is far simpler and now Deskhot is easy to spot among the other tabs.

new favicon

 

New design for the lab

A new theme for the lab gave more focus on content and separated navigation completely. Now text is a bit bigger and the centered headlines make spotting the beginning of each article much easier. The text in the sidebar was rewritten and anything non-trivial was moved in the bottom of the page.

lab theme v2

 

Releases: task management for people who create products

The problem with project management tools today is that they attempt to organize everything, making it easier for supervisors to have a global view of a production process. Releases puts the people who actually create a product in the center of attention. The single most important thing in producing any product is not work-hours or work units or anything else. It is the way the product is gonna be when it’s finally released.

By focusing on product releases you eliminate a lot of the “project management noise”. Sure, things like work-hours are nice to know but they remove your focus from what really matters. Releases purposely removes all that information so people who make the actual work can focus on what is top priority. It is currently in the early stages of development. Expect a private beta in a couple of months.

releases task managment got smarter

 

New posts

How Google wasted time and resources trying to copy Facebook and why you shouldn’t do it on
Forget the competition. Just use your products

How to create an invisible login form that you don’t have to look for on
The little details of the web

Why you shouldn’t waste your time on the iOS vs Android debate on
Native mobile applications VS the web