HTML5 Video File Conversions

I think the best part of HTML5 Video is getting to convert the source file into a billion different formats for all the browsers (note: sarcasm). For a while I was struggling with Adobe's Media Encoder, trying to get that to do my work. It never really rose to the challenge.

Recently I've been using Adobe just for the FLV/FLA files (for flash fall-back) and then using Miro's video converter for the rest. It works a lot better, but is still not fool proof. I've found that the iPad and iPhone formats are still a little hit or miss when I test on the various iOS versions out there. What I like about Miro, though, is that I don't need to tune every parameter for the mp4. All I do is pick iPad and it takes care of the rest. Beautiful.

Another option is zencoder which has a nice API you can use. This would be a great option if you had to convert uploaded video or if you had a ton of video and you wanted to programmatically convert it. Just wanted to throw that out there.

iPhone, HTML5 Video & AWS: Movie format unsupported

I've been working hard on a website that will have a video section using HTML5 video with a flash video fall back. It has taken way longer than I ever expected. What a mess this has become, trying to support every browser's own format, and then trying to support old browsers on top of that. Yikes.

One particularly troubling part has been the iPhone/iPod Touch support for video. Using multiple video encoders (Miro, QuickTime Pro, and even Sony Vegas HD) I was unable to post any video on the web that did not cause the iPhone to say "This movie format is unsupported." And it was crap, because I knew it WAS supported. And, even worse, the video would sometimes work... randomly.

Well, after lots of pain and banging my head against the wall, I found it out it was because I was using HTTPS on Cloudfront. For some reason, iOS occasionally rejects the Amazon Cloudfront SSL/TLS. Nice.

So, just take the S off of the HTTPS when playing HTML5 video served from Cloudfront.

Flowplayer Flash Video and SetReturnValue

I recently set up a HTML5 video system with flash video fallback, for those crazy people with old browsers. VideoJS.com is a great site for explaining how to do this. Use this with the site with the info here for a great solution.

Anyway, the flash fall back using Flowplayer commercial was giving me a bunch of issues on IE7 & IE8. One was there was an error that was being logged in the console about 'SetReturnValue' not being defined. Turns out, the <object> tag must have an ID. The example code I copied from VideoJs's front page didn't have this. So, make sure you have a unique id set.

The other issue had to do with Flowplayer's branding removal (with a commercial license). Turns out, the registered domain name is not for the domain the video is being played on, but the domain the swf file is loaded from. I had the swf files on a CDN and the branding was not removed. Just had to move it back to the main site and all was good.

How to fix MySql Replication Error 1236

We have some websites that use two database servers in master-master replication. Recently one of the servers died and had to be resurrected. (They are cloud based and SoftLayer doesn't seem to have their cloud-based offering thing nailed down, yet. Every week one of them will go down and be unresponsive until tech support does some magic.)

After one of the servers was brought back up the other server would not connect. It "Slave_IO_Running: No" and "Seconds_Behind_Master: null" which means it was not playing nicely.

First, I went to the MySql log files, which for this server is found at /var/log/mysql.log and looked at the last few messages here by running "tail /var/log/mysql.log" from the command prompt. This revealed the error number (server_errno=1236). It also had the following info:

Got fatal error 1236: 'Client requested master to start replication from impossible position' from master when reading data from binary log

Just before this entry in the mysql.log it indicates the file and position that it is trying to read. So, with this data in hand, I headed over to the master server and took a look at the bin logs. They are located in /var/lib/mysql/. Here I took a look at the file in question using the mysqlbinlog utility. I used the following command to check out the bin log. Obviously, you'll have to replace the position and file name with the position and file indicated in your mysql.log.

mysqlbinlog --offset=128694 mysql-bin.000013

And, this is what I saw here, among other things:

Warning: this binlog was not closed properly. Most probably mysqld crashed writing it.

Well, that explains things! When the server crashed the bin log was not closed properly. This is easy to fix. Going back to the slave server, I stopped the slave, reset the bin log position and started the server.

STOP SLAVE;
CHANGE MASTER TO MASTER_LOG_POS = 0;
CHANGE MASTER TO MASTER_LOG_FILE = 'mysql-bin.000014';
START SLAVE;

I simply pointed the slave to the start of the next bin log. It started right up with no problem.

Update 5/21/13 - An anonymous commentor made a good point about potentially losing data using the above technique. If you are in a situation like I am (master-master replication with 100's of gigs of non-critical data) this is really the only way of getting back up without significant down time. But if you are in a master-slave configuration with a manageable data set or its critical you slave doesn't miss any data, you should probably dump the master database and re-create the slave database to make sure you didn't miss anything.

SimpleXMLElement and EntityRef XML parser

I am using the PHP class SimpleXMLElement to take care of parsing some XML data that I am sourcing from 3rd parties. It had been working well for a while, but I just discovered an error that was popping up frequently. This error was "XML parser error : EntityRef: expecting ';'".

This error comes about as a result of XML input data being improperly encoded. Two data sources I was using had encoded things like "&", "<" and ">" by leaving off the semi-colon. In other words, the ampersand had been encoded as "&" instead of "&". SimpleXMLElement doesn't like this and throws a warning fest.

To fix the problem, I added a line before calling SimpleXMLElement:

$xmldata = preg_replace('/&(amp|lt|gt)([^;])/', '&$1;$2', $xmldata);
$obj_xml = new SimpleXMLElement($xmldata);

The preg_replace fixes the encoding problem and adds the ampersand for you. Just a note, this will only fix the encoding for the three characters I specified above ("&", "<" and ">"). If there are others that are causing problems, you'll have to add them to the first argument of preg_replace().

Here is another blog/article that helped me discover the underlying issue.

Computer Backups

I use Backblaze to back-up my computer's files. Let me explain why...

It is always important to have your files backed-up. And to have those files backed-up in a location that is secure. That is, away from your computer and locked and/or encrypted. There are a few big players in the online backup service sector. Carbonite and Mozy are both good examples.

It is important to have all my files backed up, especially on my work computer. If something were to happen, like fire or theft, I need to have a backup copy of all my files so I can get back to work as soon as possible. The great thing about online back-up services is that your back-up happens automatically, whenever you have an internet connection. The more automated the better!

I've been using Carbonite for the past couple of years for my laptop. It has worked pretty good. I like how they add a little icon in your file explorer on top of each folder and file that is backed up. It is a great visual cue that helps me to see exactly what is backed up and what is not.

HOWEVER, over the past few months my computer fan will turn on really loud, even when I am not using the computer. Looking closer I found that Carbonite's software was eating around 50% of my processor, causing my computer to heat up and the fan to turn on. As soon as I disabled Carbonite, the CPU utilization fell and the fan turned off. That's not right!

Well, I finally broke down and contacted their technical support. First, their technical support sucks. They have a crumby web interface, their emails are formated all funny, and the people on the other end take forever to get back to you. But, the worst part is that after I jumped through their hoops and they finally got back to me, all they said is that I use my computer "too much." Seriously!? It runs at 50% even when I am not using my computer.

What a waste of my time.

SO, now I am using Backblaze. I heard about them a while back through this Slashdot article about how they had made their hardware design public. Very cool: open-source hardware. Now that's an innovative company. And, as a double bonus, they will encrypt my files in a way that is extra safe. Only I can open my files. (I have a feeling those tech guys are Carbonite can browse my personal files whenever they'd like.) Last, but not least, when I need to restore my backed-up files, they will mail me a physical disk with all my files on it. Awesome!

And, I must mention this because I am a web designer, the Backblaze website is WAY better looking. Nice work, guys!

Storing Passwords in Databases

In the last few weeks/months there have been a couple high-profile computer breaches in the news. One was at Plenty of Fish, an online dating website. The second was at HBGary, a computer security company (ah, the irony).

Plenty of Fish made a major security mistake. They stored passwords in their database in plain text. This means, if you had access to their database (legitimately or through a SQL injection vulnerability) you could see anyone's password. And this is typically a big security no, no. Passwords should always be hashed before being stored in a database.

Hashing is a one-way encryption that prevents a password (or any character string) from being un-encrypted. So, when a user logs in, you hash their entered password and compare it against the hashed password in the database. If they match, the password is the same. It is simple AND safe.

Two particular hashes are quite popular. SHA1 and MD5. Although they are beginning to be a bit dated now and, if I remember correctly, MD5 has been shown to have some vulnerabilities. Some of the more recent hashes are SHA256 and SHA512. (Use the hash function to implement them in PHP.)

Over at HBGary, the "security" firm, they actually did use hashes to store passwords. They used the common MD5 hashing algorithm. The problem with their implementation, is that they added nothing to the password before hashing it. This is an issue because people have created massive online collections of auto-generated hashes from between 1 and 12 characters (typically). So, if you have a hash, and you're a hacker, you look up the hash in this table and (if it is a normal-sized password) it will likely be found.

The best way to make this look-up table irrelevant is to add a set of constant, known characters to the password before the hashing takes place. This technique, called adding "salt" to the password, will create an extra long password, that will never show up in a look-up table. Why? Because it would take more than a lifetime to compute look-up tables that long. Hashing is an expensive operation, in terms of CPU cycles, and the longer the original text, the longer it takes to compute.

If you are building a website, and you have passwords to store, remember: salt & hash. Mmmm, sounds tasty!