Web Development Intensive

The following is a brief example of working within a Subversion version control setup.  In this case, we will assume that some overworked server administrator (i.e. me)  has set up a repository where a large amount of source code magically is stored and backed up by Subversion, a popular version control system.  In our case, the source code is all the example files from this course.

Read the overview of Subversion concepts in the post linked above before proceeding.

The hypothetical setup

The under appreciated developers (i.e. you), just need a way to checkout (i.e. download) the latest copy of the code from the repository (i.e. the server), make some changes to some of the code, and then commit (i.e. upload) your changes back to the repository.  You do this so that we can track the changes made to the files and so that everyone else can download new-and-improved version of the source code that you created at their leisure.

Using source control keeps the source code safe from being accidentally deleted, prevents two developers from overwriting the same file at the same time, and it keeps a record of who changed what and when they did it so that your manager (i.e. me) can make sure his minions (i.e. you) are doing their assigned jobs in a timely manner.

Think of it as a backup system with benefits.

The Subversion client

To interface with our Subversion repository, we use a Subversion client application.  Just as we use web browser clients to surf web pages on the server, and we use file transfer clients to browse files on the server, we also use Subversion clients to manipulate version-controlled files on the Subversion server.

The popular client applications for Subversion provide nice graphical user interfaces for you to use so you don’t have to manually send Subversion commands to our Subversion repository on the Linux server.  Popular Subversion clients that I currently recommend are TortoiseSVN for the PC and SvnX for the Mac.  Both add extra Subversion-related options to the context menu (right click menu on PC; control-click on the Mac) within Windows Explorer and Mac Finder, respectively.

Make sure you have one of these clients installed before following the instructions below.

Getting a working copy of the code

To get a “working copy” of the code, meaning to download the latest version of all the code from the Subversion server, you make a folder on your client machine where you want to store your copy of the code.

a folder to hold our working copy of the code

Right click on the folder name and select “Checkout” from the new options that the Subversion client added to your context menu.

Subversion checkout

Enter the URL of the repository: “http://svn.onepotcooking.com/webdev/trunk” and the usual username/password in the screen that pops up.

Checkout settings

Successfully logging in will checkout (i.e. download)  a full copy of all the latest source code in the repository and put it into the selected folder on your client.  You will see that the folder and files on your computer have special icons next to them indicating their version control status.

Everything's good

The green checkmark icon means that you have not modified any of the files in this folder since last downloading the files from the Subversion server.

You’re now ready to make changes to some code.

Make some changes

Go into the source code you just checked out on your client and open up the file class12/subversion/index.html in your text editor of choice and make some changes to that file.

Update the code

You’re almost ready to “commit” (i.e. upload) your changes back to the repository.  But first you have to make sure nobody else changed the same files as you between the time you last downloaded them from the repository and the time you finished editing them.

If two people edit the same file at the same time, you wouldn’t want one person to overwrite the other’s work.  So Subversion automatically checks for conflicts.  If they are easy to solve, such as if one person edited the top of a file while the other person edited the bottom, Subversion merges the two copies of the file automatically.  If the conflicts are complicated, Subversion will as you to manually merge the two versions of the conflicted file.

So update your working copy with a fresh copy of the latest code from the repository by using the Subversion client’s update command on the project folder.  This will update everything inside of the folder and check for conflicts between the changes you have made and the changes others have made.

Subversion update

Commit your changes

Once any conflicts have been resolved, you are ready to commit your code to the repository.

Use the Subversion client’s commit option to upload your copy of the entire folder to the repository for safe keeping.  You can also just commit specific  files rather than the whole folder, but for now we’ll upload any changes we made to any files in the folder to keep things simpler.  Subversion is smart enough to only upload the files that have been modified anyway.

Subversion commit

Add a file

Now let’s say you want to add a new file to this project.  On your client computer, create a new file in the class12/subversion folder called <your_name>.html, where <your_name> is replaced with your first name and last initial, of course.

By creating this file on your client machine, it is not yet added into the version control system unless you make it so.  To add it to the Subversion repository, you need to use the Subversion “add” command.  So right-click on the file and click the Subversion add command.

Subversion add

Adding a new file is just like making any other change, whether it be adding, deleting, or modifying a file on your client.  The changes you make on your client are not uploaded to the server repository until you commit them using Subversion’s commit command.

So now go ahead and commit the changes you’ve made.

Subversion commit

The next time anyone else updates their “working copy” of the code using the Subversion update command, they will see this new file has been added.  In other words, if I don’t see your file the next time I update, I will know you didn’t follow this tutorial!  I should have forced you to use this technology from the beginning of this course.

Further changes

From this point onward, always download the latest copy of all files using Subversion’s update command on the project folder before you make major changes.  This reduces the chance that your work will have a conflict with another developer’s work, and it will reduce any chance that you are working on an outdated copy of a file.

Start-ups

The web, at barely fifteen years old, is still an open field for innovation and entrepreneurial zeal.  This is in a large-part due to the low up-front costs in starting up a web-based business.  The initial investment of many web-based businesses is relatively low, compared to traditional businesses, while the potential rewards, given the huge potential market of web visitors, are ever-increasing.  You don’t need to buy any raw materials to set up shop online – you don’t even need to be in the same country to which you sell products or services.  You just need some understanding of web development and a sharp mind for opportunity.  Of course, it goes without saying that most start-ups fail.

Start-ups, when first starting out, will generally be low on cash.  To get cash, they will usually have to raise money from investors.  To raise money from investors, they will usually have to show investors the promise of what they are developing.  So, as a generalization, start-ups will often ask people to work for less money than they can make elsewhere so that they can keep their overhead low and develop their technology enough to attract investment, at which point they will hopefully be able to pay people adequately.  Instead of cash, they will often offer employee  stock options, meaning that employees have equity and stand to gain (sometimes hugely) from any future company profits.

To sum it up: make sure that whatever you stand to gain is worth the lower pay and higher risk than other forms of employment.

Agencies

Web developers need not be entrepreneurs.  Since the late 90′s, interactive agencies have popped up left and right to bring the marketing and advertising skills perfected at ad agencies of the past into the new domain of the internet.  Many of these agencies were originally print ad agencies which have transitioned into the web.

Going with the old ad agency model, these businesses have transformed the originally ad-hoc process of building websites into a results-oriented industrial assembly line process.  As in any industrial manufacture, web or interactive agencies divide interactive accounts into specialized work roles, where each person plays a small part in building and promoting brands on the web.  And it’s not just agencies – any medium to large corporation will have a web department handling a variety of functions, often in the same streamlined process (but by no means always in this way.)

Agency work is a good way to experience a wide variety of different types of websites within a very short period of time.  Agencies also have a regular flow of employees going in and out, meaning that it is sometimes easier to find work with agencies than with other, less dynamic, companies.  The pay is usually decent and standardized.

Large commercial brands will often hire agencies not because they are especially creative, but simply for the reason that they are reputable, large, experienced, usually heavily insured, and familiar with dealing with many types of clients.  Agencies often subcontract their work to more specialized firms who do design or development for them.  There is a whole feeder industry surrounding agencies.

Careers

Here is a short list of web-related jobs that one often sees advertised at interactive agencies, corporate web departments, and web design shops:

• Creative Director
• Interactive Director
• Product Manager
• Project Manager
• Web Marketer
• Web Producer
• Account Manager
• Web Strategist
• Content Strategist
• Web Marketer
• Media Planner
• Media Buyer
• Web Analytics Specialist
• Community Manager
• Information Architect
• User-Experience Designer
• Web Developer
• Front-End Developer (client side)
• Back-End Developer (server side)
• Flash Developer
• Flex Developer
• Web Designer
• Graphic Designer
• Q/A Engineer

Pay

A contract web developer can make anywhere between $15-$150/hour.  On the low end, you find newly minted developers with very little experience, as well as outsourcers from countries with a low cost of living.  On the high end, you get experienced consultants with years and years of experience and a proven track record of results with many former big-name clients.

As full-time employees, web developers can again make anywhere along a wide range of salaries: from $30k to $180k.  It depends, as with anything else, on experience and knowledge of the industry, as well as the type of company you work for.  Non-profits and start-ups tend to pay less than interactive agencies, but they often attempt make up for that gap with perks such as interesting work, good benefits, idealistic projects, stock options, etc.

What to charge your first client

Not a lot.  Explain to your first few clients that you are trying to gain experience, and be fair in your pricing.   You want to be able to develop accurate estimates of how long various types of jobs would take you, and price accordingly.  But to start with, things will take you a long time, and so it may be better to price your work as a flat fee for the entire project, rather than by the hour.  Give your clients a break, so long as they are lenient with you.

Once you get comfortable with how quickly and well you are able to complete jobs, which will happen sooner than you think, start raising your prices.

Recruiters

Many technology and design recruiters and staffing firms exist to match job applicants up with companies in need of specialized help.  You should be aware that recruiters generally charge clients much more than they pay their employees.  It is not unusual for them to take a 20-50% cut.  So if you are hired by a recruiter to work a job for $60/hour, you should not be surprised to learn that the recruiter is charging the client $100/hour and keeping $40 for themselves for every hour you work.

If you use a W-4 tax form with the recruiter, where they withhold taxes on your behalf, they may provide insurance, retirement plans, and other perks similar to those of a full-time employment.  However, if you have a corp-to-corp relationship and file your income from the recruiter on a 1099 form or similar, you do not get those perks, and you may be able to use that fact to negotiate a slightly higher rate since the recruiter’s expenses in hiring you will be significantly less.

Jobs sites

There is no replacement for networking anywhere and everywhere.  However, there are some popular jobs sites as well.  Here is a list of a few I know of:
http://newyork.craigslist.org
http://hotjobs.com
http://monster.com
http://dice.com
http://simplyhired.com/
http://elance.com

Trends

E-commerce is still in its infancy.  To those of us who have ordered all our holiday presents online  for years, it may seem like e-commerce is a saturated market.  But in fact, industry analysts predict massive growth in e-commerce in the coming years as more and more businesses move onto the web and new markets open up as more and more people become web-enabled.

For years, the industry has also chattered a lot about convergence – the meeting of markets like e-commerce, mobile devices, publishing, telephony, television, and web marketing.  With the explosive growth of web-enabled phones like Blackberry, iPhone, Android devices, VoIP, Kindle, and P2PTV, the web as we previously knew it is becoming more and more intertwined with the mobile device and with other broadcast technologies.  This may explain the enthusiasm with which investors and technologists embrace otherwise inconsequential products like Twitter which seamlessly bridge the divide between web and mobile.

An interesting artifact of the lowering cost of mobile and web-based technologies is the developing world’s embrace of mobile phones, and SMS (text messages).  Since many developing countries lack the telecommunications infrastructure required for high speed land lines for internet connectivity, and given that mobile phones in many places other than the US are very cheap to begin using, more and more people are communicating and accessing information through mobile phones and SMS, which work on cellular and satellite networks.  These cell towers and satellite systems bypass the more traditional methods of accessing the internet that we in the US are accustomed to, such as DSL, cable, or T1 lines.  This difference in access may have interesting implications for the development of the web for a more global audience.

The use of online technology for education is also growing every year as more schools offer distance learning programs to attract wider student bodies.  And childhood education has embraced technologies like online resources and electronic whiteboards that allow students to interact with a teacher’s console using handheld devices as they find that under-performing students fare better when taught with technological devices.

Google

Google makes money almost entirely through advertising.  Almost every product and innovation they release is geared towards bringing more traffic to their search engine and analyzing traffic on all of their sites for metrics which can be used to sell targeted advertisements to their users.  By providing extremely userful applications like Google Maps and GMail, Google drives users towards their search engine and their advertisements while collecting data on users’ interests and habits to better refine their targeted advertisements.

Web Advertising

Web advertising is traditonally priced in one of two ways:  CPM or CPA.   CPM stands for Cost-Per-Thousand – it means the price an advertiser is willing to pay to have 1,000 people view their advertisement.  CPA stands for Cost-Per-Action, sometimes also known as Cost-Per-Acquisition, or Cost-Per-Conversion.

It used to be that advertisers paid sites like Yahoo, Microsoft, and Google on a CPM basis to run their ads.  However, in recent years, advertisers have seen a drop in the number of people who view an ad and then actually click on that ad.  The click-through rate of web advertisements has dropped so much that today, only a few tenths of a percent of people will actually click ads they see on the major web portals.  Estimates vary, but click-through rates for web ads are said to be anywhere between 0.1% to 0.3%.

So many advertisers have switched to the CPA model, where the amount they pay depends on a more concrete metric, such as how many viewers perform a particular action that results from the advertisement.  An “action” in this context could be anything, such as registering for a site, signing up to receive more information, or buying something.  The term “conversion rate” refers to the percentage of viewers of an ad who end up taking such a follow-up action.  Web marketers spend a significant amount of their time worrying about these numbers.

Targeted advertisement is all the rage, since a site that can collect the habits and preferences of a visitor can then run advertisements that are more likely to appeal to that visitor. They can therefore charge more of the advertisers since the click-through and conversion rates are likely to be higher.

Facebook & Twitter

Nobody really knows how Facebook, Twitter, or similar popular social network services will become profitable if they ever do.  But the major social networks have hundreds of thousands to hundreds of millions of users, which means they have a huge number of eyeballs looking at their sites every day.  The potential for advertising revenue to those eyeballs is great, especially considering how many users have voluntarily indicated preferences and affiliations with their favorite brands.  Facebook is thought to perhaps be already be profitable based on its heavy advertising.  This may explain some social networks’ high stock valuations.  However, such companies, which seem to rely on buzz and the good-will of their users, are extremely wary of alienating those users by overrunning the sites with advertisements.  Hence you will see them trying to advertise in roundabout ways, such as Facebook’s Beacon service, or selling user data to third parties rather than using it themselves for targeted advertisements.

By offering additional services like mail, applications, photo sharing, and such, Facebook is hoping to prevent people from abandoning their site in favor of the next big thing, as happened to its social network predecessors like Friendster and to a lesser extent, MySpace.  The more effort people put into curating their online personalities and profiles, the less likely they are to abandon the site and thereby waste all that effort.  I would not be surprised to see Twitter start doing the same.

Some people argue that Facebook’s valuation may be an indicator of investors’ faith in its viability as a “Web OS”, an operating system that runs entirely on the web and upon which you can build more sophisticated web-enabled social networked software products.  Maybe the future will see Facebook transform into the next Microsoft.

Mobile

It should be obvious that the mobile web is the next big thing.

Now that the course is over, you are free to keep using the class database at your own risk for as long as you like.  However, I will be making changes on the server in preparation for the next course, and these will most likely affect your database tables.  So I recommend you make a backup of your database tables, as explained here.  If you eventually set up your own hosting account with your own domain, you will be able to import these tables into the MySQL server, as detailed below.

Exporting your tables

To export a copy of your tables from the database, follow these steps:

Step 1. Select the database that holds the tables you want to export.

Do this by selecting “classdb” in the top left of the page.  This will take you to the “Database View” that will show you a list of all the tables in that database.

select database

Step 2. Select the “Export” tab at the top of the “Database View” page

export tab

Step 3. Select the tables you want to export

Do this by holding down the Control key and clicking the name of each database you want to select in the list. (On Mac, hold down the Command key instead of the Control Key).

select tables to export

The default settings should be fine for the rest of the controls on this part of the page.  The important points are to make sure both the Structure and the Data of your tables are getting exported.  So the checkboxes next to the words “Structure” and “Data” should be checked.

make sure Structure and Data are both selected

Step 4. Click the “Save as file” checkbox

At the bottom of the “Export” page, there is a checkbox next to the words, “Save as file”.  Make sure this is checked.

In this same area, you can optionally select to have the database export file compressed into a zip or gzip file.  This will make the file smaller, which is perfectly okay, but not necessary.  It’s your choice whether to compress the file or not.

save as file settings

Step 5. Click “Go” to export

At the bottom right of the “Export” page is the submit button with the word, “Go”.  Click it to complete the export.

go!

Step 6. Save file

After a few seconds, you may see a dialog box asking you whether you want to “Open” or “Save” the file.  Make sure you select “Save”, and as always, make sure you remember which folder the file is getting saved into on your hard drive.

save file

Congratulations!  You have successfully exported your tables.

Importing your tables

Assuming you have exported your tables from the classdb database, you may one day want to import them your own database on your own server.  Doing this is even simpler than exporting was.

Before you begin, you must have a database set up.

It’s important to remember that we only exported the tables, not the actual database.  So before you can import your tables, you will have to first create a database in which to import them.  You can call this database any name you want.  It does not have to be named, “classdb”, like the original database.

Most web hosts will have a control panel that allows you to create databases on your server.  Usually they disable this feature from phpMyAdmin.  As an example, here’s a screenshot of what the “Create a database” control panel looks like with Dreamhost, the host for the server we used in class.

create database in Dreamhost control panel

Assuming you have created a database on your server, or you are using an existing database, here are the steps necessary to import your tables.

Step 1. Select the database that you want to import the tables into

Do this in phpMyAdmin by selecting your database name in the top left of the page.  This will take you to the “Database View” that will show you a list of all the tables in that database, if any exist.

select database

Step 2. Select the “Import” tab at the top of the “Database View” page

import tab

Step 3. Select the file with the tables you want to import

The “Import” page will ask you for the “File to import”.  Click the “Browse” button to browse your hard drive for the file you exported.  Once that file is selected, you can leave all the other settings at their defaults.

file to import

Step 4. Click “Go” to import

Click the “Go” button to complete the import.

go!

Congratulations!  You have imported tables into your database.

As a confirmation, you should now see the new table names show up on the left side of the page, where all tables are listed that are contained within the selected database.

updated list of tables

Here are some of the most common Subversion commands that you may one day run on the server, which uses a Linux operating system.  Most of the time, your SVN client, like TortoiseSVN or SvnX, will handle these things for you.  But here are the commands for your reference anyway, in case you one day consider yourself something of a server administrator.

Note 1: These examples use “$SVNREP_DIR” in place of the full path from the server root to the folder on the server where your repository is stored.  And “myproject” is in place of your project’s name.

Note 2: Also, these examples show how to run the commands on the same server that stores the repository.  If for some reason, you are running them on a different computer than the one that stores the repository, you can replace the file:///somefolder/ urls with an absolutely http://someurl.com/somefolder/ path.

//command to create a repository
svnadmin create $SVNREP_DIR/myproject

//command to create the layout of the repository
svn mkdir -m "layout creation" file:///$SVNREP_DIR/myproject/trunk file:///$SVNREP_DIR/myproject/tags file:///$SVNREP_DIR/myproject/branches

//do an initial import of your project into the repository
cd ~/myproject
svn import -m "initial import" . file:///$SVNREP_DIR/myproject/trunk

//check to make sure all files were imported
svn ls file:///$SVNREP_DIR/myproject/trunk/

//move your original project folder to a backup folder
mv myproject myproject.origin

//checkout the project from the repository
svn co file:///$SVNREP_DIR/myproject/trunk myproject

//make sure the files checked out successfully
ls myproject

When projects require several developers (i.e. designers, developers, html coders, etc), they may run into situations where multiple people try to edit the same file at the same time. This could be problematic, since one developer could easily overwrite another’s work.

To prevent one person from overwriting another’s work, most teams of coders use some form of version control. Version control software allows multiple people to simultaneously work on the same files while not overwriting each other’s work.

Subversion is probably the most popular version control software:
http://subversion.tigris.org/

You can find the Subversion online manual here.

The Repository

Subversion uses a type of client/server architecture.  The permanent location of all files is on the server, in what is called a Repository.

The Repository has the current version of all files, as well as all the older versions of the files stored in a nicely organized way. It keeps a permanent record of any changes to the project.

Clients can always get the latest copy of the files by copying them from the Repository.  So developers generally make changes to files on their local machines, and then upload them to the Repository so that other developers can get the lastest version of the files.

Internally, the Repository in Subversion has three subfolders: “trunk”, “branches”, and “tags”.

The “trunk” is the main folder that always has the latest copy of the main version of the code.

The “branches” folder is used if you decide to fork your code and simultaneously work on two or more different variations (or branches) of your code.

The “tags” folder allows you to give nicknames to prior versions of the code for easier access later on: for example, you may decide to call revision #12 the “gold_release” with a tag so that it is easier to remember than a number.

In general, for a simple project, you will work with code in the “trunk”, unless you are an advanced user.

The Client

Your local computer runs a Subversion client that automates the common tasks that you will need to do when using version control. The recommended client for Windows is TortoiseSVN. For Mac, you should check out SvnX.

Any client will easily allow you to run the basic Subversion commands:

• checkout - this command is used to get all the files from the repository at once and download them to your client.  You usually run it once at the beginning of development to set up your working copies of the files
• import - this command is used to import files as a batch into the Repository.  It is usually used once at the beginning of a project that has already begun without version control.
• add - this command adds a file to the repository.
• update - this command updates all the files on the client machine with the latest version from the Repository
• delete - this obviously deletes files
• commit - permanently store any changes in the Repository.  Note: you need to commit after every modification, merge, add, or delete in order for those changes to be permanently stored in the Repository.

Working Copy

When a developer downloads the latest copy of the files from the Repository, they are placed in what is called a Working Copy. This is exactly what it sounds like – a copy of the original files that a developer can use to work on her client machine.

Once a developer has finished modifying the files, she has to then upload them back to the Repository so other developers can see the changes.

Checkout

When a developer starts working on a project that is using version control with Subversion, the first thing they will need to do is get a copy of the latest version of all the files in the project.  This is called the initial checkout.

Any Subversion client, like TortoiseSVN or SvnX, will have an option to checkout the files in a repository.

This command is usually only run once to do the initial setup of the Working Copy on the client machine.

Update

To get the latest copy of files from the Repository, a developer runs a command called Update. This automatically downloads the latest files from the Repository.

Developers should run an update immediately before editing any file, and again immediately before Committing any changes to the Repository.

Commit

When a developer has finished making changes to her Working Copy of the files, she first runs the Update command to make sure that nobody else has modified the same files she is about to upload, and then runs a Commit command that automatically uploads the changed files to the Repository.

Conflicts

When two developers have modified the same files at the same time, and then they both Commit their changes, the second developer to commit his changes will experience a Conflict. Usually the work the developers have done is in different parts of the file, so Subversion can automatically merge the two versions together.

However, in some cases Subversion cannot figure out how to merge the two versions of the file, so it asks one of the developers to manually merge the two. Subversion will show you the difference between the two files, and you can usually easily see what has to be done to get the two versions to merge correctly.

Once you are finished merging, you must of course commit any changes in order for them to be permanently stored in the repository.

Workflow

To sum it up, here are the usual steps you will take when working on code that is held in version control:

1. update to the latest version of code
2. modify the code
3. update again to check for any conflicts from other people’s changes
4. merge any conflicts that have arisen from other people’s modifications
5. commit your changes