Yosemite and SSL Certificate Verification Failed with cURL

Let's say you're a developer, like myself, and you have just updated to Apple's latest operating system (Yosemite or 10.10) and all of a sudden the Twilio PHP SDK stops working. And, it gives you an error something like this:

SSL: certificate verification failed (result: 5)

The problem disappears when you comment out the CURLOPT_CAINFO option, which makes it a bit more confusing. In other words, you can't use your own CA bundle/file. Others have also noted that it only really happens with certain certificate types, like wild-card domains (e.g., api.twilio.com doesn't work, for example).

Well, it seems to be a bug on the specific version of cURL, version 7.37.1, that Apple has bundled with their operating system. Version 7.37.0 and the latest version of cURL both work fine.

Check this out for all the details: http://sourceforge.net/p/curl/bugs/1404/

I haven't found a good way to fix it yet, myself, but the above link does offer some suggestions.

Fixing json_encode() Problem in PHP 5.5 on Ubuntu 13.10

I upgraded to Ubuntu 13.10 over the weekend and everything went pretty well. It automatically brings PHP 5.5 with it, too. When I tried to run a web app I was working on, though, I received this message:

Fatal error: Call to undefined function json_encode()

I subsequently Googled around and came across a highly-ranked blog article which makes it sound like PHP 5.5 drops support for json_encode because of some silly licensing argument. The truth is, PHP 5.5 does NOT drop support for json_encode. For a much better and much clearer explanation, I recommend reading this article.

Now, enough of the balleyhoo! How do we fix it?

It was was easy! Install the php5-json package and restart Apache. Voilà!

sudo apt-get install php5-json
sudo /etc/init.d/apache2 restart

Note that the author of the blog article linked above mentions that re-installing the php5-dev pacakged fixed his woes. While that didn't help me, you may want to give it a try if the above didn't help you.

Symmetric-Key Encryption: AES in PHP and what "mode" to choose

Doing some encryption? Want to use the same key for encryption and decryption? Well, I did too. Here are the results of a lot of internet searching.

The 'mcrypt' extension in PHP is the place to go for AES, symmetric-key encryption. AES is safe and secure – the US government has even ok'd 192-bit AES (and up) for top secret documents. You can install it by running yum install php-mcrypt (if you're running Red Hat or CentOS) and then it is available for your coding pleasure.

The next decision is what "mode of operation" to use for your AES encryption. There are a few different ways to do all the fancy math and permutations. Some are more secure than others. You'll notice that when using the mcrypt_encrypt() command, you have to specify the mode as the 4th parameter. You have a choice to make. PHP 5.4 currently has the following modes available (you can find all the options available to your environment by using mcrypt_list_modes):

• cbc
• cfb
• ctr
• ecb
• ncfb
• nofb
• ofb
• stream

If you're in the same boat I'm in – aka "not a cryptographer" boat – then this is just a nice list of letters. But there are a lot of cryptographers who put a lot of good info on the interwebs, thank goodness. I ran across this article on AES usage tips, then there is the overly-neutral Wikipedia article, and a good-enough answer to a similar question on Stack Exchange.

Here are my collected thoughts on the matter:

1. One glaring thing I came away with from all these sources is this: DON'T USE ECB. 
2. The second thing I came away with is this: CTR would be the ideal, if it was available. This is because it does some authentication to make sure the encrypted message really is an encrypted message and not some sort of cryptographic trojan horse. (The simple solution to this is to make sure you HMAC hash your encrypted text to make sure it isn't tampered with.)
3. The last thing I came away with is: Use OFB or CFB because they are pretty darn good and available in mycrpt.

Other thoughts? Do you know of even better articles on modes of encryption? I'm all ears!

Checking the last SQL statements used by PHPActiveRecord

I'm using PHP-ActiveRecord on a project and am loving it. But there are some things to learn and as a guy coming from the write-your-own-sql world, I do occasionally want to check what SQL queries that ActiveRecord is sending out there. I've used this bit of a code a few times to do so and it is immensely helpful for me as I try to see if I'm using this package correctly:

echo Model::table()->last_sql;

Pop that in after you do your Model::find() or Model->save() or whatever, just be sure to swap 'Model' out with your model's actual name.

Installing Multi-Byte PHP Functions

I received this error today:

Fatal error: Call to undefined function mb_strlen()

And it would look the same, no matter what multi-byte PHP function I might be using: mb_strpos(), mb_substr(), etc. And it is beginning to be important to have this extensions installed if you are working with international, multi-byte character sets like UTF-8.

The fix is super easy. All you need to do is install the PHP multi-byte extension with yum (if you're on CentOS or RHEL). Do the following from the command line:

yum install php-mbstring
service httpd restart

That installs the extension and restarts Apache. Then you're good to go!

Installing Memcached on CentOS

I have a web server running CentOS and wanted to get some good caching going for PHP use. This is how I installed memcached, from the command line:

• Install memcached
• yum install memcached
• Install Pecl (if you don't yet have it, installing pear installs pecl)
• yum install php-pear
• Install Pecl's Memcached package (for PHP)
• Install dependencies, as needed
• yum install php-devel
• yum install gcc
• yum install zlib-devel
• pecl install memcache
• Use a text editor to add "extension=memcache.so" to php.ini (may be in the /etc folder)
• Restart the service
• service memcached start 
• service httpd restart

And when you want some raw stats, you can get them via telnet:

• telnet localhost 11211

And type "stats" to see hits, misses and more!

Here are some examples of memcached use in PHP.

SE Linux and "Can't connect to MySQL server"

I've been setting up a server for the last few days for production. It is to be highly secure, so it is a real pain. Anyway, I ran into this problem where I could not connect to the remote database. It was driving me nuts. (I'm using CentOS by the way, and I was setting up a web server to connect to a remote MySQL database server.)

First, I checked that I was getting network connectivity by doing

telnet 10.0.0.1 3306

from the command line (where 10.0.0.1 is the IP of the database server). And, I got some gobbledygook that had the words "MySQL" in it, so I know that worked and the network was there.

Then, I checked the database's user list, to make sure connections were allowed from the user at my host. I did this by going into MySQL's CLI and entering

SELECT `Host`,`User` FROM mysql.user WHERE 1;

This printed out a list of all the users so I just made sure it and the host were correct. (There are more MySQL troubleshooting tips here.)

Then it finally hit me. What always causes problems for hours on end? SE Linux!! I know it is good for my server, but it sure does drive me nuts sometimes. If you want your web server to be able to connect to a remote DB, you have to give it rights with SE Linux. Run this at the command line:

setsebool -P httpd_can_network_connect_db on

and for good measure:

setsebool -P httpd_can_network_connect on

There, all fixed!

Update: HA HA! I've had this problem before and completely forgot about it.

PHP UTF-8 Script Input Cleaner

When you switch to UTF-8 on your website, there is a few things that everyone recommends (for good reason) like using multi-byte functions (e.g., mb_strlen()) and adding the HTTP-EQUIV attribute header. One of these commonly recommended things is that you should clean up all user-submitted input.

With UTF-8 comes the ability to submit a lot of crazy characters to the script, either by POST or GET. These crazy characters might be control characters, invalid UTF-8 characters or some other charset that was mixed in for good measure. So, I created the following function to help clean my inputs:

<?PHP

function cleanUTF8(&$input, $stripSlashes = true) {
 if ($stripSlashes) $stripSlashes = get_magic_quotes_gpc();
 if (is_array($input)) 
  foreach ($input as $k => $v) cleanUTF8($input[$k], $stripSlashes);
 else {
  if ($stripSlashes) $input = stripslashes($input);
  $input = mb_convert_encoding($input, "UTF-8", "UTF-8");
  $input = preg_replace('!\p{C}!u', '', $input);
 }
}

?>

It is a recursive function in that it will iterate into a variable if arrays exist, as they sometimes do. It will also strip slashes for you if your version of PHP still has magic quotes on.

It removes invalid characters through the mb_convert_encoding() function. Anything that is not UTF-8 is dropped. Lastly, the fancy preg_replace() function removes all control characters. (The \p{C} means "all control characters", the !! are delimiters the same as // or ##, and the last "u" modifier means "treat this as UTF-8.")

At the top of your script add this to iterate over the input array and clean up the data:

<?PHP

cleanUTF8($_POST);
cleanUTF8($_GET);

?>

SimpleXMLElement and EntityRef XML parser

I am using the PHP class SimpleXMLElement to take care of parsing some XML data that I am sourcing from 3rd parties. It had been working well for a while, but I just discovered an error that was popping up frequently. This error was "XML parser error : EntityRef: expecting ';'".

This error comes about as a result of XML input data being improperly encoded. Two data sources I was using had encoded things like "&", "<" and ">" by leaving off the semi-colon. In other words, the ampersand had been encoded as "&" instead of "&". SimpleXMLElement doesn't like this and throws a warning fest.

To fix the problem, I added a line before calling SimpleXMLElement:

$xmldata = preg_replace('/&(amp|lt|gt)([^;])/', '&$1;$2', $xmldata);
$obj_xml = new SimpleXMLElement($xmldata);

The preg_replace fixes the encoding problem and adds the ampersand for you. Just a note, this will only fix the encoding for the three characters I specified above ("&", "<" and ">"). If there are others that are causing problems, you'll have to add them to the first argument of preg_replace().

Here is another blog/article that helped me discover the underlying issue.

PHP to Remote Database Error: "Can't connect to MySQL server on..."

On a new server migration project I am working on I have separate web and database servers. So, I use a private network to connect them. It took me a day to get working, so I am putting up a post to help others in similar situations.

First, just to clarify, I knew it was a web server problem (and not a network or db permissions issue), because I could connect to the MySQL server just fine from the MySQL command line on the web server. But, when I pinged the PHP web page from my browser I'd get this error:

Can't connect to MySQL server on '10.1.1.1' (13)

The problem was SELinux, which was installed and crackin' down on Apache rights.

One simple fix:
setsebool -P httpd_can_network_connect=1

More information can be found in this nice article:
http://www.ehow.com/how_2090983_connect-remote-database-under-selinux.html

SELinux would allow me to connect from the command line just fine. I could even execute the PHP script from the command line and connect fine. But SELinux did not like it when Apache tried to connect to a computer on the network. Hence the policy update, as stated above.

If you have other issues or it appears that your db server is having issues, here are some other MySQL debug tips as well.

Add Text to an Existing PDF Using PHP

I am finishing up on a project right now that fills out a PDF application for a merchant account. The software helps the user through the application process by asking more targeted questions and filling in fields that are known. In order to do this I had found two PHP libraries that work quite well.

The first is FPDF. It is a free PHP library that allows you to create PDFs. It is not a module or binary code, just a straight up PHP script library. Color me impressed. You can download it at http://www.fpdf.org/

The second bit of code you need is called FPDI. This will import a PDF as a template and allow you to overlay text. It worked quite well for my project as I was taking an existing PDF and filling in fields on top of it. Just like FPDF, it is super easy to install because all it is is a collection of PHP script files. You can download it at http://www.setasign.de/products/pdf-php-solutions/fpdi/

Here is a quick demo by FPDI on how to use these two modules together.

To make it all work I created a flat, PDF version of the template PDF. I removed all of the form fields using Adobe Acrobat. The first run I got an error: "probably uses a compression technique which is not supported". This was easily fixed by saving the PDF as PDF/A using Adobe Acrobat. (Update below.)

If you have any questions, please let me know. I'd be glad to share more of what I learned.

Update (10/22/10): I've learned that saving the PDf as a PDF/A doesn't always solve the error I explained above. What has worked, however, is running Adobe Pre-Flight on the document. If you go to "Advanced" > "Print Production" > "Preflight" and then click on "Analyze and fix", Adobe will work some magic on the document and make it work with FPDI.

Update (1/27/11): FPDI, the free version of the PDF parser, requires the PDF to be version 1.4 or below. To save a PDF in this format, open the document in Adobe Acrobat, click "Save As" and then select the type "PDF, Optimized" then click the "Settings" button and select "Adobe Acrobat 5.0". This will save the PDF in version 1.4, which was last used in Adobe Acrobat 5.0!

Random exclamation points inserted into emails

I use PHP mail() to send out emails quite often. I recently found out that if you create your email as one long string without line breaks, you'll get random exclamation points in your emails. I was recently pointed towards RFC 2822 section 2.1.1 for the solution. It appears that no line can be longer than 998 characters without a line break.

How do you fix this? (1) The first way is to base-64 encode the data (best solution for plain text emails to retain long lines). Check out this blog for some info on that. (2) The other, much easier way, is to simply insert line breaks in your email (best solution for HTML emails where long lines are not needed). Add a "\r\n" to the end of each line in your email to make sure you never go over 998 characters.

PHP image manipulation

If you have ever wanted to dynamically create or edit images, GD is a great solution for PHP (and LAMP configurations). I am writing this blog entry mostly because I was amazed at how easy it was to install. I just picked it in easyApache (using WHM) and "tada!" – it works. I can't say that about imagemagick. I've tried and tried to get that bit of software to work (on multiple servers) and it is always a pain. I remember on my first server (hosted by another company) I had requested imagemagick and they even had trouble installing it—the help desk ticket was pretty long by the time it was closed.

Anyway, moral of the story: Go with GD for dynamic image manipulation in PHP.

APC.php not Working

I was getting a new server up and running and wanted to install APC (Alternative PHP Cache). Things were going great. I even ran phpinfo() and noted that APC had an entry. Everything was super, right?

I wanted more confirmation, so I used SSH, located apc.php and copied it to a domain for public access. I wanted to know how well APC was working for me. However, it wouldn't run. It sent back errors and when I tried to debug it I got no where.

Turns out, APC wasn't really doing much of anything. I had PHP installed and running as a CGI instead of an Apache Module. If PHP runs as a CGI, APC closes at the end of each script run and the cache is lost—it doesn't make much sense to use APC in this situation. While runing as a CGI has some security benefits you lose a bit of performance and, more importantly to me, APC doesn't really work.

I use WHM and so I went to "Main >> Service Configuration >> Apache Configuration" and then changed the PHP 5 handler to DSO. This solved the problem and apc.php loaded just fine the next time I tried. And it even had some pretty stats for me to see.

MySQL and mysql_select_db

I was convinced that mysql_select_db() was not working today. I wanted to simultaneously connect to two different databases. However, when I called the mysql_select_db function, it seemed that both connections were changing to the database I specified. I couldn't get one link to stay on the original database while I changed the second link to a new database.

After a little digging around I found a helpful user comment about using mysql_connect. The fourth parameter of this function is "new_link" and if you want to simultaneously have two connections open and both connections use the same credentials, you must set this parameter to TRUE. If you don't, it sees that the credentials are the same and will reuse the same resource link, even if you set a different variable.

So, this:

<?PHP
$conn1 = mysql_connect(HOST, USER, PASS);
mysql_select_db(TABLE_1, $conn1);  
$conn2 = mysql_connect(HOST, USER, PASS);
mysql_select_db(TABLE_2, $conn2); 

turned into this:

<?PHP
$conn1 = mysql_connect(HOST, USER, PASS, true);
mysql_select_db(TABLE_1, $conn1);  
$conn2 = mysql_connect(HOST, USER, PASS, true));
mysql_select_db(TABLE_2, $conn2);

And tada, I could simultaneously connect to two (or more) databases.

Starting another script

I have a PHP script that customers would use to start a process. The process was time-intensive, so I had it start another script in the background. It was important that the customer did not have to wait until the long process ended. After trying a few things, I found a forum that gave me my solution (sorry, can't find where it is anymore). This starts a PHP script in the background from within another PHP script:

shell_exec('php /public_html/accountname/longProcess.php > /dev/null 2>/dev/null &');

Simply replace the script name with our own and it should work (note: not sure if the whole path is required). What this does is force the standard output and error output to be discarded and the final ampersand causes it all to be run in the background.

Text Area to HTML Editor

TinyMCE is my visual HTML editor of choice (a.k.a. What You See Is What You Get or WYSIWYG). I use it on a number of websites to allow customers to update content without needing to take a class on HTML and CSS. It has worked pretty good for me thus far and best of all, its free.

One particular feature that I've recently discovered has made me pretty happy. TinyMCE allows me to supply a list of URLs for TinyMCE to make available when a user wishes to insert a hyper link. This is a beautiful thing. Before the user had to copy and paste every URL they wanted to link to.

Just as cool is that I can also supply a list of images and videos that can easily be inserted (no copying and pasting these URLs either). So after uploading an image to the server, they can edit a webpage and TinyMCE already has this image's URL ready if it is to be used.

In order to do all this, at the end of my TinyMCE initialization script I added these three lines:

<script type="text/javascript">
 tinyMCE.init({
  ...
  external_link_list_url : "/tiny_mce/links.php",
  external_image_list_url : "/tiny_mce/images.php",
  media_external_list_url : "/tiny_mce/media.php"
 });
</script>

Now, on the other side of these specified URLs I have a PHP script that rounds up every URL (or image or video) on the server and provides it in a JSON encoded list for TinyMCE. This way, when a user wants to insert a link (or image or video), TinyMCE already has a good list of links that the user can choose from.

Let's assume, for the PHP example script, that you have a 2D PHP array called $linkOptions that had every link the user might want to use. Here is a part of links.php, specified above:

<?PHP
 header("Content-type: application/javascript");
 
 echo "var tinyMCELinkList = new Array( \n\t";

 $firstLink = true;
 for ($j = 0; $j < sizeof($linkOptions); $j++) {
  // Go through each link of the array
  if ($firstLink) {
   echo "\n\t"; // No comma for the 1st
   $firstLink = false;
  } else echo ",\n\t";

  echo '['. htmlentities($linkOptions[$j][0]); // Link Name
  echo '","'. htmlentities($linkOptions[$j][1]) .'"]'; // Link URL
 }

 echo "\n".');'; // All done

Then, you would need to do replicate something like this for each of the link lists specified. Hope you find this feature as useful as I. Good luck!

PS. TinyMCE is also the HTML editor used for WordPress.

jQuery's getJSON

I recently had a struggle getting jQuery's $.getJSON function to work on Internet Explorer (IE6 & IE7). I tried and tried to debug the javascript, but was getting no where. After giving up for a while and coming back to it the next day, I thought of checking the target file—the file jQuery was fetching asynchronously—for problems.

It turns out that IE doesn't like JSON encoded documents that are specified as UTF-8. Not sure what the problem is, but the document has to be returned in a format IE likes. If not, IE will not allow jQuery to process it. Now, I had a header on this file that specified the content-type and charset. But, I changed it and instead I put only the content-type at the top of my JSON-returning PHP files:

<?PHP
header("Content-Type: application/json");
?>

Why does IE balk at UTF-8 specified charsets for JSON-encoded responses?? Who knows. But, from the research I've done, specifying the charset in the HTTP header is not something that is required. I am sure there are good reasons to specify the charset, but this (above) works great for me.

email piping with PHP and cPanel

I had a lot of trouble piping emails to a PHP script today. There was a real scarcity out there in terms of information (or I just didn't know how to search for the right terms). So, I thought I'd post about my fun in case someone else got stuck. (By the way, I use cPanel 11 and started with this script.)

What I wanted to do was set up an email address that would pipe to a PHP script. That is, my PHP script would get a copy of the emails. Then the PHP script could parse the file and do whatever; make a database entry or something similar.

I ran into two problems.

First, I would get a return email that said "Local delivery failed." It would have this at the top of the email:

pipe to |/home/accountname/public_html/emailin.php
   generated by msg@accountname.com
   local delivery failed

I found some website somewhere that suggested changing the file permissions for the PHP script (emailin.php in this case). So that is what I did, and it seemed to help. I used the cPanel File Manager to change the permissions to 754.

That fixed this problem.

The second problem was that despite receiving the email at the script and doing the appropriate things with it, I would still get an error email back. So, even though it all worked, my email would bounce back saying:

A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:

 pipe to |/home/accountname/public_html/emailin.php
   generated by msg@accountname.com

The following text was generated during the delivery attempt:

------ pipe to |/home/accountname/public_html/emailin.php
      generated by msg@accountname.com ------

Again, after a few hours of searching I found someone that suggested changing the hash bang at the top of the script file to this: #!/usr/bin/php -q

That dash q made all the difference.

This was my adventure.

Sitemaps

Sitemaps are always a little fun. You get to see just how many pages your site has and which ones are more important to you. But, they are really important for search engines. It helps the automated search spiders find all the pages you have and then they can gain a helpful perspective based on the last update and your priority ranking.

This is the website to check out if you want to learn more about Sitemaps: http://www.sitemaps.org/protocol.php

I usually do my sitemaps in PHP. So, I am going to break new ground here and drop some code on my blog. Since most of my sites are now PHP, here is what I typically use:

<?PHP header("Content-Type: text/xml; charset= UTF-8"); ?>
<?PHP echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n"; ?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
 <url>
  <loc>http://www.mydomain.com/</loc>
  <lastmod><?PHP 
     echo date ("Y-m-d", filemtime("index.php")); 
     ?></lastmod>
  <changefreq>monthly</changefreq>
  <priority>0.5</priority>
 </url>
 <url>
  same stuff here...
 </ulr>
 
 ... and add more urls as needed
 
</urlset>

Make sure you test the URL in your browser first, but then it should be good to go. You can let the search engines know about it in the webmaster tools or even add a reference to it in your robots.txt file ("Sitemap: http://www.mydomain.com/sitemap.php").