High Performance Blog

Today one of my scripts prompted the #2013 MySQL error while querying a huge innoDB table (31GB in ~154million rows). Some queries worked, some just failed.

Looking at the logfiles, I saw the following message:

InnoDB: Page checksum 1840120551 (32bit_calc: 1224736073), prior-to-4.0.14-form checksum 1811838366
InnoDB: stored checksum 3031359782, prior-to-4.0.14-form stored checksum 1811838366
InnoDB: Page lsn 47 623631862, low 4 bytes of lsn at page end 623631862
InnoDB: Page number (if stored to page already) 68664,
InnoDB: space id (if created with >= MySQL-4.1.1 and stored already) 22
InnoDB: Page may be an index page where index id is 35
InnoDB: (index “PRIMARY” of table “tracking”.”banner” /* Partition “p4″ */)
InnoDB: Database page corruption on disk or a failed
InnoDB: file read of page 68664.
InnoDB: You may have to recover from a backup.
InnoDB: It is also possible that your operating
InnoDB: system has corrupted its own file cache
InnoDB: and rebooting your computer removes the
InnoDB: error.
InnoDB: If the corrupt page is an index page
InnoDB: you can also try to fix the corruption
InnoDB: by dumping, dropping, and reimporting
InnoDB: the corrupt table. You can use CHECK
InnoDB: TABLE to scan your table for corruption.
InnoDB: See also http://dev.mysql.com/doc/refman/5.5/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.

I tried to check the table -> mysql failed.

I tired to recover the table -> mysql failed.

The filesystem as well as the RAID were in a healthy condition and even rebooting the machine didn’t to the trick (would have been odd if it did). So, I asked my friend Google, and Google pointed me to an interesting post:

I ran into a problem where, when dealing with HUGE tables (location tables for http://Stiggler.com ), there was an innodb page error, and mysql would try over and over to repair it, and would inform me that it could not repair it (and would then try again, etc).

[...]

I put the force_recovery mode to 1, then restarted mysqld, exported the entire database (i expected to get an error when it got to the bad table, but i never had a problem). After dumping the database, i removed the force_recovery option from my.cnf and restarted the service, and after a few moments, it started back up, and the problem was gone.

I can’t remember where exactly I found this quote (sorry!), but what this guy was basically trying to say: Dump your database and wait for the magic to happen!

So I did, tried my query again and it just works! I dunno what magic voodoo effect dumping a database with mysqldump does, but it just works. Really, this seemed like the most stupid approach to fix this issue, but it did the trick!

Next time you hit the “#2013 Lost connection” issue.. try dumping your database!

Today one of my scripts prompted the #2013 MySQL error while querying a huge innoDB table (31GB in ~154million rows). Some queries worked, some just failed. Looking at the logfiles,...

Today a customer asked for automated screenshots of his website. So first thing to do was asking Google how it could be accomplished on a Linux WebServer. Most of the results referred to installing an X-Server, using Firefox and stuff. This sounded a bit tricky and quite frankly.. over the top.

The solution I came up with is (IMO) much easier and less prone to failures: WKHTMLTOPDF + Imagick Convert.

WKHTMLTOPDF uses QT with a Webkit Widget – it renders HTML pages inlcuding JavaScript like Apple’s Safari Browser. So creating a PDF from a website became very easy:

Usage: wkhtmltopdf -L 0 -T 0 -R 0 -B 0 http://mywebsite.com/ test.pdf

That’s all you need to create a PDF from your website including all images, javascript rendered contents and such. The parameters “-L -T -R -B” define the the border margin of the PDF document. The default value is 10mm – I prefer zero.

Now that we have a PDF of the website, we need to convert it into a JPG file. This can be acomplished using ImageMagick.

Usage:  convert -density 600 test.pdf -scale 2000×1000 test.jpg

In case your webpage is pretty long it might happen that your PDF contains multiple pages. Imagick generates one JPG file per page named test-0.jpg, test-1.jpg and so on. All you need to do is combining the images with Imagick in a second step, which is also pretty easy and straighforward:

Usage:  convert test-* -append single.jpg

That’s it… now you have a single image file from your website.

Today a customer asked for automated screenshots of his website. So first thing to do was asking Google how it could be accomplished on a Linux WebServer. Most of the...

Just a quick note for everyone who’s using FFmpeg for creating screenshots from video files. Today I noticed that FFmpeg can be VERY slow on large/long movie files, but there’s a pretty neat trick to speed up the screenshot generation.

I used to create my screenshots this way:

ffmpeg -i /var/www/input.mov -y -f image2 -ss 1234  -sameq -t 0.001 “/var/www/screenshot.jpg” 2>&1

and on huge files it sometimes took minutes. The solution is simple:

ffmpeg -ss 1234 -i /var/www/input.mov -y -f image2  -sameq -t 0.001 “/var/www/screenshot.jpg” 2>&1

Put the -ss parameter in front of the input file and FFmpeg skips to the selected frame almost instantly.

hf!

Just a quick note for everyone who’s using FFmpeg for creating screenshots from video files. Today I noticed that FFmpeg can be VERY slow on large/long movie files, but there’s...

Some say, node.js helps you to make your applications scale. In fact, it’s really fast and flexible. Check out this realtime browsergame, build with node.js.

Some say, node.js helps you to make your applications scale. In fact, it’s really fast and flexible. Check out this realtime browsergame, build with node.js. Share|

Learning how to scale isn’t easy without any prior experience. Nowadays you have plenty of websites like highscalability.com to get some inspiration, but unfortunately there is no solution that fits all websites and needs. You still have to think on your own to find a concept that works for your requirements. So did I.

Continue reading

Learning how to scale isn’t easy without any prior experience. Nowadays you have plenty of websites like highscalability.com to get some inspiration, but unfortunately there is no solution that fits...

It might sound like a trivial task to cache Smarty templates, but when your website starts growing you should consider some more advanced strategies of caching your content using Smarty.

Continue reading

It might sound like a trivial task to cache Smarty templates, but when your website starts growing you should consider some more advanced strategies of caching your content using Smarty....

In our office we are currently evaluating the pros and cons of Magento. Besides all the great features it offers it has a huge con – the performance. We read a lot about it but couldn’t find any reliable statements about how fast or slow it really it is. So I put a plain Magento install on my testing machine and gave it a shot..

Continue reading

In our office we are currently evaluating the pros and cons of Magento. Besides all the great features it offers it has a huge con – the performance. We read...

Let’s start with the basic setup for my test scenarios. In our office we had an old and dusty, but still working Intel Xeon 3305 left. It has 2Gigs of RAM and a 1TB SATA harddisc. It’s nothing special, but still good enough for some benchmarks.
Continue reading

Let’s start with the basic setup for my test scenarios. In our office we had an old and dusty, but still working Intel Xeon 3305 left. It has 2Gigs of...

as if there weren’t enough blogs out there in the wide wide web,  I had to share my thoughts too. Hopefully you won’t regret wasting your time here and enjoy a few of my thoughts and experiences that I’m gonna share here… every once in a while.

This blog is supposed to deal mainly with tuning websites, written in PHP. I’m gonna share my benchmarks, A/B tests and everything else I’ll come across. I really love getting the last few requests per seconds out of a machine, using op-code caches, memcached, static file caches, lighttpd, nginx – just anything that’s performing well nowadays.

Just a few words about me…

I’m working as a web coder for about 10 years now. I started with Perl in the late 90s to add dynamic contents like News on mostly static old-school HTML websites.  Over the years I switched to PHP, created websites from 0 users and 0MB traffic per month, up to 2.5Mio users and 12TB of traffic a month. At a certain point when a website starts growing the performance, and later scalability, becomes one of the most concerning and important points. If you’r wasting  CPU cycles, you’r wasting money!

Subscribe to the feed and stay tuned!

as if there weren’t enough blogs out there in the wide wide web,  I had to share my thoughts too. Hopefully you won’t regret wasting your time here and enjoy...