Here are some tips to scaling well and enabling your website to grow large, easily. (By the way, I am assuming a LAMP configuration.)
- 1. Be very careful using session data (e.g., $_SESSION['varname']), because if you ever need to expand to multiple web servers, this wont work. Session data is stored in memory and will not be present if the web visitor would happen to use a different server for a second request.
- 2. I know this is common practice, but I will repeat it anyway: Use library and include files for your scripts. In these files include common functions and common parts of your rendered pages (e.g., headers, menus and footers). This makes updating your entire website as simple as editing one file.
- 3. Plan ahead for major traffic spikes by using grid hosting (e.g., Media Temple's "Grid Service"). Grid hosting is a deal where server resources are pooled together and used for whatever site needs them. This way, when you get a major spike in traffic (e.g., you get slashdotted), you have ample processing power and bandwidth to handle it. You may have to pay a bit more when you get traffic spikes, but it is better than being suspended by your host.
- 4. Deliver your media content on another set of servers. Media files like movies and audio can take up a lot of bandwidth and file space. Whenever possible offload these files on to someone else's site and you'll save some cash and be more scalable. For example, upload your movies to YouTube or Vimeo and then embed their flash players in your own site. Alternatively, you can use a Content Delivery Network or something like Amazon's S3. When you host content with any one of these solutions you also gain in the area of speed and quality of delivery.
There is a lot more that can be said in this area, but these are the things I've come across that have helped me.