How to Shape the Mobile Data Market

(Part 3 of "Who Will Pay for Mobile Data?")

There's a big nasty dilemma hidden at the heart of mobile computing:  No one knows how we'll pay for all that mobile data we're supposed to use in the next few years.  The question doesn't get much publicity, but it drives some of the most intense debates in mobile, including net neutrality and the wireless bandwidth "crisis."

This is the conclusion of a three-part series on the issue. In Part 1 (link), I talked about the tech industry's unlimited vision for the growth of mobile data, and why I think it won't come true because we'll run out of people willing to pay for data service

In Part 2 (link), I discussed the alternate scenario, in which everyone is willing to pay for mobile data and adoption of it continues to accelerate.  In this case, the mobile operators will need to invest urgently in increased capacity, and even with that investment we'll eventually run out of wireless bandwidth. 

The two scenarios leave mobile operators trapped between the need to expand their networks and the fear that they won't be able to pay for the expansion.  So the operators are trying to get other parties to help pay for the network.  I believe that's the real driver behind the net neutrality debate and the rhetoric about a wireless bandwidth "crisis."  Ultimately, government regulators will decide who will pay and how the mobile data network is structured, which will have a huge effect on which companies win and what we can do with the network.

In this part I'll give my take on what we should do about the situation, and I'll talk about the opportunities all of this change creates for operators, handset companies, and developers.


The look of mobile data in the future

If you only took away two messages from the first two posts in this series, these are the ones I'd want you to remember:

1. The only thing we can predict for sure about the future of mobile data is that it's unpredictable.  Maybe I'm right that it'll saturate soon; maybe Cisco's right that it'll go on growing explosively for years; maybe we'll average out to something in the middle.  The variables in play are so numerous, and so complicated, that absolutely no one can predict for sure what will happen.

In that sort of uncertain situation, I think our top priority should be to keep the mobile market as flexible as possible, so it can respond quickly and efficiently to whatever the customers decide to do.  That means we should ensure that market signals -- things like pricing and customer demand -- are as clear and unambiguous as possible, so we'll all know what the real level of demand is, and we can all respond to the same base of information.  The word "transparency" gets overused these days, but goodness gracious we need as much transparency as possible in mobile data.

2. We should plan wired and wireless data together.  We need to deal with the reality of the mobile network and market, not what we might want it to be.  And the reality is that we're not creating a separate wireless data network, we're creating a single integrated wired and wireless network.  A lot of the political rhetoric about mobile data talks about a completely cellular data future as some sort of public goal.  It's more like a public fantasy.  Every forecast I've seen from the wireless operators requires that they be able to offload a lot of traffic to the wired network.  Forget about wireless replacing wired; what we need to do is make sure they both work together well, with each focusing on what they do best.  That means wired is used whenever possible because in most cases it's cheaper and higher capacity, while wireless fills in the gaps.

We should set up a level playing field between wired and wireless so the market can sort out which traffic should go where.  Artificial political goals for the penetration of wireless, or favoring one network technology over another, are incredibly dangerous because they may lock in a market structure that turns out to be unaffordable.  In fact, because the market is so unpredictable, those sorts of goals are almost certain to be wrong.

So I get queasy when the US Federal Communications Commission, and even big companies like Google, argue that wireless data should have different regulations than wired data.  I think that increases the risk that we'll accidently bias the overall network in the wrong direction.


What we should do

As I've said before, I am not a big fan of government regulation in business, because it's usually inefficient and slow.  However, there are some situations in which you can't get the government out of the market, and I think cellular wireless is one of those cases because the public ultimately owns the airwaves in most countries.

So if we're going to have government regulation, let's do it right. 

The grand bargain.  The operators are asking for some mammoth benefits.  In the US, some of the biggest operators want to merge.  Okay, let's let them do it.  I don't think TMobile US is large enough to be viable in the long term anyway, so we need to merge it with either AT&T or Sprint.  If TMobile joins AT&T, which is the current proposal, the next merger in the US will be Verizon-Sprint; I think we have to accept that as well, for the same reason. 

The operators in the US and Europe want more spectrum allocated to them.  Again, I'd go ahead with it.  In the US, the television networks aren't using the extra spectrum, so it ought to go somewhere useful.

But in return, we should demand serious changes in the cellular data market.  I'm not talking about tweaks at the edges, I mean permanent changes in the rules of the game, designed to ensure lasting competition and a more flexible market that responds better to customer needs.

Here's what I propose:


Stop whining about the wireless "crisis" 

The first step is to change our rhetoric.  The bandwidth "crisis" is the tech industry's equivalent of the War on Terror: it's based on a genuine problem, it can never be completely solved, and it can be used to justify many actions that people might not otherwise consider.

The idea of a wireless crisis is an incredibly convenient tool for motivating government regulators.  Elected officials assume they are responsible for solving a wireless spectrum crisis, since they allocate wireless spectrum.  If it were called a "Verizon and AT&T don't want to pay for a bunch more cell towers crisis," I don't think President Obama would propose spending $50 billion on it.

This isn't just a US issue.  Anything one government does in mobile data is played back in other countries as a justification for equivalent actions there.  On a recent trip to Australia, I was surprised to hear a radio commentator complaining at length about the government's plan to supply broadband service to many Australians through landlines rather than wireless.  You can make a good argument for using landlines, since (as we discussed in part 2) they can carry a lot more data than wireless.  But the commentator was upset that Australia was failing to do "what Barack Obama is doing in the United States."

It's reasonable to ask what's so wrong with a little crisis hype and international competition.  After all, governments move far too slowly in most cases, so if a bit of alarming rhetoric makes them respond faster, isn't that a good thing?  The trouble is that we'll all have to live with the results after the "crisis" is "solved."  In that world, no matter how much spectrum we allocate to wireless data, service will continue to have slowdowns, outages and service gaps, especially in the United States, because it's more profitable for the operators to run their networks right at the edge of overload (in this sense they have the same financial incentives as airlines). 

We're lying when we tell people that the whole wireless data network could collapse.  Although service problems are a certainty, there is virtually zero risk of a full network collapse, unless the operators cause it themselves by underpricing data plans and selling more smartphones than they can support.  And we're misleading people when we say that prices will go up unless we allocate more spectrum.  Prices will eventually go up no matter how much spectrum we allocate to data, because demand for cellular data is growing faster than supply.

By overstating the risks and talking about the "crisis" as a temporary, fixable thing, we create an unrealistic public expectation for the quality and price of cellular data in the future.  That may well be advantageous for a couple of quarters or even a year, but in the long run it will erode public trust when we don't deliver the benefits we promised.  The wireless operators, especially AT&T in the US, already have big image problems.  Overpromising will make the problems worse.  To the extent that government agencies, and mobile tech companies like Apple and Google, participate in the crisis rhetoric, they risk their credibility as well.

We need to ask ourselves as an industry if we want to have the same sort of public image in five years as the airlines have today.  If not, we should be honest with people now.  For example, I think there is a convincing, legitimate case for reallocating old TV spectrum for data services.  Without it, mobile data prices will go up faster, and a lot of the features many of us want from mobile data may not be affordable.  But we should also be honest with people that cellular bandwidth overload is a chronic disease rather than a crisis, the network is not going to collapse unless we're incompetent, cellular service will not be as fast or cheap per bit as a wired, and cellular data will generally be a supplement to our wired broadband, not a replacement.


Make the cellular data market transparent

The problem with the cellular data market as it's structured today is that it often hides from users the real cost of the network they use, so they can't make well informed choices, and it's hard for us to tell which buying patterns are genuine and which ones have been created artificially.  For example, the cost of your smart phone is subsidized, so you don't realize what an expensive piece of hardware you're carrying in your pocket.  You're told that you have unlimited data, but actually if you use it too much your operator will probably reduce your data speed without telling you. 

By making cellular data seem cheaper than it is, we encourage people to use the network more, increasing the very overload that we're supposed to be fixing.  Some of the proposals for the future of mobile data would further increase the overuse of cellular data by making it seem even cheaper to users.

The structure of the mobile market also limits competition among mobile operators (especially in the US), and reduces competition between mobile phone manufacturers.

I think this systematic distortion of the market must stop.  If people could see the real cost of cellular data, they would make better-informed decisions about when and how to use it, and we wouldn't need secret back-end controls on traffic.  Meanwhile, more competition in services and phones would mean faster innovation, more consumer choice, and more efficient prices.

Here are some specific steps I think we should take:


1.  Ban covert traffic limits.  Today some wireless operators (and some wired ones as well) are quietly reducing the quality of service they deliver to some users, without telling them.  This is done through various techniques including "traffic shaping" (prioritizing or delaying certain types of data packets) and "throttling" (reducing the throughput of the network, or the speed of certain transactions).  In effect it usually means reducing the connection speed of people or apps that use the network the most.  For example, Dean Bubley recently wrote about an ISP who consistently reduced data throughput at particular times of the day (link).

There are some types of traffic management that make sense.  E-mail spam can be reduced through throttling that limits the number of e-mails that can be sent by a single account per second.  Throttling can also be used to limit malware attacks, by reducing the ability of a rogue app to flood the network with traffic.  And I think it's fine to enforce the speed you paid for in your Internet connection.  For instance, if you've paid for a 10 MBPS connection and the operator limits your throughput to 10 MBPS, I do not have a problem with that.

But in some cases the operators are limiting network performance to covertly restrict users, either by interfering with certain types of traffic, or by limiting the speeds of some users without telling them.  For example, the current Verizon Wireless terms of service give them the right to reduce the throughput in your "unlimited" data plan if you're in the top 5% of data users (link).  They can do this without notifying you.

This sort of hidden restriction is damaging to the market because people may sign up for a wireless plan believing they will get more service than they actually will.  They can't make a fully informed decision between wired and wireless service because they don't know how much wireless data they're really going to get.  This may misallocate resources and make the wireless network even more overloaded than it would be otherwise.

The answer to this is simple: Require operators to notify a customer when they have throttled or shaped his or her service (other than enforcing the promised speed of the connection).  I am not against throttling in general, but it should not be done without notification.  A text message would be fine.  The Internet speedometer, which I discuss below, will also help with this problem.


2. Require a data gas gauge and speedometer in smartphones.  Can you imagine buying a car that didn't have a gas gauge and speedometer?  That's essentially what we do today with smartphones.  For most smartphone users today, there is no easy way to tell how much data throughput you're getting from the network, and how close you are to any limits on your data usage.  Some operators bundle apps to do this, some have more arcane ways to check, and some send you a text if you get close to the limit.  But I think it's fair to say that most people are in the dark about their usage until they get their monthly bill, and if they do go over a limit they will have trouble figuring out why.

This is an easy problem to fix.  We should require that every smartphone have an app, accessible at the same level as the Settings app, that tells the user how close he or she is to hitting any data caps in the service plan (for example, if you are a Verizon user, how close are you to getting throttled?).  The app should also show how much data you're using at any particular time, so you can see how much throughput the network is really giving you. 

We also should modify the signal strength bars to change color depending on how much data you're consuming at any moment.  This would show you when you're using a website or app that uses huge chunks of data.  When customers see that video or Flash makes their signal bars turn red, they'll be much more cautious about using those sites on the wireless network.


3.  Decouple the phone purchase from the network.  Currently in the US and much of Europe, if you sign a contract for a data plan, you get a discount of several hundred dollars on a new phone purchased at the same time. But you have to buy the phone through the mobile operator, giving them huge control over the selection and features of the phones they sell.  Basically, users are not free to pick the phones they want; they have to take the phones their operator chooses to sell.

This operator lock-in is subject to all sorts of backroom manipulation.  Weak phone vendors are forced to comply with a huge list of tests and requirements, while for stronger vendors the rules are often waived.  I've also been told privately by some operators that they deliberately discriminate against some handset vendors because they just don't like them.

The handset vendors aren't completely clean either.  A vendor with a hot handset may restrict its availability to a single operator in order to extract concessions from them.  Can you say iPhone?

It's a wonder that some operator or handset company hasn't been sued already for restraint of trade.  With the amount of operator shelf space shrinking in the US due to mergers, I think it's only a matter of time before there's a legal detonation. 

In addition to the legal risk, these restrictions have the effect of restricting customer choice and competition, so they are bad for transparency.  It's time to open up the handset market.  To make that happen, subsidies should be separated from the purchase of a particular phone.  When someone signs up for a plan, they should get a voucher for a discount on any phone.  The voucher can be used at that time to buy a phone in the operator's store, or it can be used later to buy a phone in any other store. 

This would encourage more selection and competition in mobile phones.  It would create more direct competition between operator service plans.  And it would put the wireless and wired networks on an even footing (can you imagine a wired data provider limiting the brands of PC that you can use with your cable data connection?).

In the US, I think we should consider one other step to open up the handset market.  In most of Europe, and many other parts of the world, there is a vigorous retail market in mobile phones sold separately from an operator.  Because everyone is on the same network standard, and because all the phones use SIM cards, it is easy to buy a new phone at retail and pop your card into it.  You do lose the subsidy, but virtually all customers know they can at least switch phones if they really want to.  This leads to a much larger selection of phones, and to higher competition between operators because it's easier to choose separately the phone and service plan you want.

The US market is much less open.  Most mobile phones are sold only through operator stores, and it can be very hard to switch from one operator to another because they have different network technologies, and some of them don't even use SIM cards.  Because it's so hard to switch phones, I think most US mobile users are barely even aware of what a SIM card is, and how to find it in their phone (most of them would probably confuse it with the SD card).

To open up the handset market, the US should require that all mobile phones use SIM cards, and that they be switchable between the major operator networks.  That way someone could go into a consumer electronics store, buy the phone they want, and use it with any network.  This will have to be phased in over time, but we're already moving toward it anyway.  Verizon and AT&T are both moving to LTE, and there are very strong rumors that Sprint will do so as well.  So some day we'll have one standard cellular technology base in the US.  In the meantime, we'll have to buy dual-mode phones that use both LTE and either GSM or CDMA, depending on which operator you use.  But the chipsets for smartphones are increasingly capable of handling several different networks, so they can switch between LTE, GSM and CDMA.  I think it would be reasonable to require that future smartphones sold in the US be SIM-based and capable of operating on all three standards.  I think the real question is how quickly we could phase in that requirement; if you have thoughts on that please post a comment.


4. Enable toll-free apps and websites.  As I discussed in Part 1, we need the data equivalent of a toll-free phone call, in which a website or mobile app company would pay for the data traffic generated by a particular app or site.  This requires changes to the operators' billing infrastructure, but I think it will be essential for enabling the growth of mobile data.  It should be an extremely high priority for the operators, and it's in the interest of web and app companies to get together with the operators to define standards for these charges, so they'll be easy for developers to work with.  I suspect there's an important role government regulators can play in helping to encourage these negotiations.


5. Do not allow the operators, or the web companies, to discriminate against one-another.  I agonized over this one a lot.  The operators would like to be able to charge web companies extra if they want reliable delivery of data (for example, in a time-sensitive app like video streaming), or if they want a guarantee of a certain level of throughput.  I understand why they want to do this, because it would help pay for their infrastructure, and I do not think it is inherently evil.  But I think it would cause too much collateral damage to the mobile market.  In fact, I think it would put us on a road toward wrecking mobile data.

The first problem is that hidden back-end charges like this are essentially an invisible subsidy for cellular data.  A user won't know the real cost of the data he or she is using, and this could end up increasing traffic on the cellular network artificially, contributing to data overload.

There are also big practical problems with implementing charges for quality of service.  As Dean Bubley has pointed out repeatedly (link), there are huge drawbacks to this sort of approach.  To give one example, there is no way to guarantee quality of service when you don't know how overloaded a particular cell site will be.  If one high-priority video session comes in, does the operator shut down five other "regular" data sessions to make way for the high-priority one?  In that case, the "regular" customers are not getting the service they paid for, and they won't even know it.  They'll just think something is wrong with the web app they're using.

I agree with Dean that there's no way to make a system like this work predictably and fairly.  Better to just charge users for the data they consume, let them know how much that costs, and allow them to adjust their own usage patterns.

The other reason we should ban quality of service fees is because in some cases they could produce in a destructive power struggle between operators and websites, with users caught in the middle.  US cable television is a nightmare example of what not to do. 

In cable television, it's common for network operators and content companies (the cable channels) to pay each other for services.  For example, Home Shopping Network reportedly pays cable TV companies to be included in your service package, because they know they'll make more money if they're seen in more homes.  They are, effectively, subsidizing your cable television service. 

On the other hand, many of the most popular channels charge the cable companies a fee for the privilege of carrying them.  For example, ESPN (the leading US sports network) reportedly charges cable companies about $4 per month per household; other popular channels are in the 5-20 cent per month range. 

The same sorts of things could happen in the mobile web if the operators could charge websites for service.  For instance, what if Facebook started offering video streaming as part of its services?  If the mobile operators tried to charge Facebook for its network usage, what is to stop Facebook from turning around and demanding a fee from the operators for allowing them to carry Facebook? 

Unless we're very careful, we could end up with a situation in mobile similar to the one in cable TV, where users get caught in disputes between the network operators and the content creators.  Some of those arguments in the US have been incredibly ugly, with users tied into long-term contracts for cable service but unable to access the channels they thought they paid for.  And remember, in cable we get these messes even though we have only have about a hundred channels to negotiate.  On the web, you have literally millions of them.

The operators should not kid themselves that they would win in this sort of showdown.  If Facebook cut off its traffic to Sprint's servers, what would happen?  Would users abandon Facebook because it's not on the Sprint network -- or would they switch off of Sprint because it doesn't have Facebook?  I think we all know the answer to that: there would be crowds holding pitchforks and torches outside the Sprint stores.  The websites have far stronger brands and far more user loyalty than the operators.  So it's unlikely that the operators will really be able to coerce money out of the most successful websites.

In practice, I think the operators would be able to get fees only from small startups that don't have brand awareness with users.  That becomes a barrier to entry for those companies, which historically have been the source of most online innovation.  To give a real-world example of what that could do to the web, look again at cable television programming: A small number of networks dominate the selection of channels, resulting in slow innovation and reduced choice.  There is very low turnover in these channels. 

If the web worked like cable TV does, we'd all still be using AOL for e-mail.

I've talked with people at small startup cable channels, and they are incredibly bitter about the barriers they face getting placement on cable systems.  They're actually counting on the web to let them bypass the cable operators.

I think the only way to make the mobile market work efficiently is to make the payment mechanisms as clear and visible as possible.  Make users pay for the data they use, and allow web and app companies to make their sites and apps toll-free if they want to, but don't start creating hidden layers of fees and subsidies.  That will just distort the market and expose operators to retaliation.  My operator friends, this is a war you cannot win -- so don't start the battle.

To formalize this settlement, government regulators should ban both operators discriminating against websites or types of traffic, and websites withholding their content from a particular operator or network.


6.  Encourage open WiFi.  As I mentioned above, we're not creating a standalone cellular network, we're creating an integrated wired and wireless network.  WiFi has a critical role to play in that network, and we should make it even more central.  Here's a question for you:  How often have you tried to find an available WiFi network, and seen no networks at all in range?  I can't speak for other countries, but it almost never happens to me in any populated part of the US.  But how many times have you tried to sign onto WiFi and found only locked access points?  That happens to me all the time. 

We already have a very dense, well-populated wireless front end to the data network in most places that matter, but we can't use it fully because most of the access points are locked down.

There are good reasons for the lockdown.  If you leave your WiFi router open, it can be hacked (actually, it can also be hacked if you keep it locked, but that's a topic for a different post).  Also, in the US if someone downloads child pornography or does something else illegal on the Internet, the law often goes after the router owner because that's the only person they can find.  You can read some horror stories here.

But getting those connections opened up would have huge benefits for the public, because it would take some of the pressure off cellular wireless.  Rather than telling people to close off their connections, we should be encouraging them to leave them open.  Regulators could help this in a couple of ways:

--First, we should require that the next generation of WiFi routers have a pass-through feature enabling public access to the Internet without giving access to the user's home network.  Traffic from the user's private connection should have priority over the public one, and if public usage is excessive the user should be able to throttle it.

--Second, the law should be changed to protect people whose open wireless connections are abused without their permission.


Opportunities

So that's how I think the future of mobile data will look: unpredictable growth, always skating the line between overloaded and overpriced, and with a huge variety of users, almost all of them with some sort of limits on their data service, and many with budget plans that encourage very careful use of data.  For the health of everyone involved in the market, I hope we'll also get regulations that make the market more transparent, and more open to new players.

It's a different mobile data world than many analysts have been predicting, but that's not necessarily a bad thing.  Often the best business opportunities happen when conditions change unpredictably.  I think this is one of those times.  So I'd like to conclude by recapping the big opportunities as I see them...

For handset vendors, I think the most interesting new opportunity will be the smartphone designed for people with limited data budgets.  How do you entice people into gradually using more data?  This is an opportunity to do a fundamental rethinking of the smartphone user experience.  Since different people will probably respond to different data features, I think it will also be an opportunity for smartphone vendors to stake out their own market segments, helping to insulate them from the intense commodity price pressure we're likely to see in generic smartphones as the market fills up.
   
Try to think like an automobile vendor in 1950.  Do you want to compete with everyone else in midsize sedans, or would you like to dominate a smaller segment like station wagons or sports cars?

To target a segment, you'll need to hire people who know how to design integrated hardware-software systems rather than just devices, and you'll need to learn to partner closely with app and web companies as peers (rather than the serf-overlord relationships you're used to having).

In the last couple of days I've been contacted privately by some people who predict even more revolutionary moves by the handset companies, most notably the idea of selling a phone at retail bundled with airtime that you've bought from an operator.  In other words, the phone comes with its own network service.  That's what Amazon did with Kindle, and there's nothing in principle to prevent a handset company from doing the same thing. 

I think there would be a lot of implementation challenges, most notably keeping access to that third party network if it starts to run out of capacity.  But it would be intriguing to see what someone like Apple would do with this.

For operators, I think it's important to pick your battles.  Although covert traffic-shaping and charging websites for service is very seductive, in the long term that will lead you into intense conflicts that you're not likely to win.  It would also create more incentives for the handset companies to set up their own virtual networks, which really would transform your networks into dumb pipes.

I think it's better to focus on new business models that are a win for both you and your business partners.  The most appealing of these to me is toll-free data.  That would be intriguing to a lot of web and mobile app companies, allowing you to build cooperative alliances with them.  And it's a whole new revenue stream that might become very large over time.

For web and app developers, the emerging segmentation of mobile data makes the idea of "enticement" even more important than it is today.  How do you give people some software for free and then entice them into paying for add-ons or other apps?  Already most of the mobile app developers I talk to are thinking along those lines, and obviously that business model is very well established on the web.  But as smartphones reach down to more price-sensitive people who are less enthusiastic about data, there will be intense demand for apps and websites that can entice them into starting to pay for bits of mobile data. 

These "data on-ramp" apps are not always intuitively obvious, and will probably differ by country (for example, mobile horoscopes were a major driver of beginning data use in parts of Asia).  The companies that can find the on-ramps will be incredibly valuable to investors, handset companies, and operators.


What do you think?

That's my take on the situation. What do you agree and disagree with?  What else would you add to the picture?  How does it differ in your country?  And most importantly, what do you think the opportunities are?  Please post a comment and share your ideas.

The Truth about the Wireless Bandwidth "Crisis"

(Part 2 of "Who Will Pay for Mobile Data?")

There's a big nasty dilemma hidden at the heart of mobile computing: No one knows how we'll pay for all that mobile data we're supposed to use in the next few years. The question doesn't get much publicity, but it drives some of the most intense debates in mobile, including net neutrality and the wireless bandwidth "crisis."

This is the second of a three-part series on the issue. In Part 1 (link), I talked about the tech industry's unlimited vision for the growth of mobile data, and why I think it won't come true because we'll run out of people willing to pay at the current rates for data service

In this part, I will talk about the alternate scenario, in which most people are willing to pay for mobile data and adoption of it continues to accelerate. In this case, the mobile operators will need to invest urgently in increased capacity, and even with that investment I think we'll eventually run out of cellular bandwidth.

This means the operators face two conflicting possible futures. In one, growth is about to slow down and they don't need to invest in a bigger network. In the other, they need to invest urgently in additional network capacity. For telecom execs, it's a bet-your-career choice with no clear winner. So, naturally, they are trying to get someone else to pay for the investment. That's the real cause of the rhetoric about a wireless "crisis," and it's driving much of the net neutrality debate.

To understand why this is happening, let's start with a look at the physics and economics of a cellular data network...


Mobile data doesn't scale like fixed-line broadband

When mobile operators in the US and Europe first built out their 3G networks, they miscalculated what people would do with them. They expected that new, relatively low-bandwidth mobile services like a simplified version of the Internet (called WAP) and picture messaging (MMS) would be the dominant source of data traffic on the network, and they structured it accordingly. But those new data services failed to take off, and the operators were left with a ton of excess capacity. Desperate to generate any revenue from their new networks, they offered fire-sale data plans for the newly-emerging smartphones. It didn't matter if the operator made a good profit off a smartphone data plan -- with the network already built and sitting idle, any data revenue was better than none at all.

So the operators in many regions gave us low-cost or unlimited data plans. Those plans set customer expectations for how they should use mobile data and what it would cost in the future.

Overcapacity continued on most mobile networks until the launch of the iPhone in 2007. We tend to forget about it today, but the iPhone was the first smartphone to make PC-style browsing practical and attractive for most smartphone users. The result was an explosion of mobile browsing, and almost overnight mobile data networks supporting the iPhone started to go from overcapacity to overload.

The reason for the overload was simple -- people in the developed world learned to browse first on their PCs, most of which have high-speed wired connections to the Internet. Bandwidth on these connections isn't infinite, but it's large enough that activities like file sharing and watching videos are mainstream.

When people started doing PC-style browsing with their smartphones, they brought their PC browsing habits with them. Unfortunately, the cellular wireless networks don't have nearly the same data capacity as the wired networks. So mainstream browsing behavior on a PC turns out to be excessive browsing on a smartphone, especially if you use a lot of YouTube.

Even if you're not a big video user, the normal sorts of messaging and web traffic created by a PC can overload a wireless network. A typical PC has a more or less continuous connection to the web, so instant messages and web app updates can ping back and forth constantly. But to conserve battery life and stretch network resources, a smartphone doesn't talk to a cellular network continuously; it basically says hello to the network, sends a message or a bit of data, and then says goodbye. Each little message and each app ping creates its own set of hellos and goodbyes. Send too many and they can overwhelm an operator's servers. Web apps send too many.

The operators, of course, can add additional wireless capacity to cope with the increased traffic, and they have been doing so. Cisco estimates that the total capacity of the world's wireless networks will increase by about 10x from 2010 to 2015. But these additions eventually run into physics problems. There's only so much information you can squeeze into a certain amount of wireless spectrum. At some point the cellular infrastructure overloads, and as an operator you have some ugly choices:

--You can add a lot more cell towers, reducing the size of each radio cell and therefore increasing the number of devices you can support. Unfortunately, this is extremely expensive -- not just to build the towers themselves, but for the fiber optic cables connecting them, the servers to manage them, and for the lobbying you must do to overcome political opposition to additional towers.

--You can offload data traffic to local wired connections. The operators are already pushing this hard, via WiFi. Some are also encouraging the installation of femtocells in individual homes and businesses. (A femtocell is basically a micro cell tower in a box the size of a wifi router. It gives you cellular service inside a building or business, using wired broadband to communicate back to the cellular network.) But that too is expensive: at around $200 a pop (link), it would cost about $13 billion to attach a femtocell to every one of the 67 million consumer broadband lines in the US. Plus there's the support cost for installing them, and the expense to put millions more cells in businesses, and the cost to buy the back-end servers necessary to support them. Nevertheless, some very smart people watching the mobile data market believe that femtocells are essential to the future of mobile data (link). Cisco estimates that offloading of some sort will handle about 20% of mobile traffic by 2015, and up to 40% in some countries.

--You can buy more spectrum, but there are huge licensing costs associated with that, not to mention the cost of retrofitting your cell towers for the new frequencies and replacing all of the phones in the installed base.

Unfortunately, even if you make all of the changes above, there are some very convincing arguments that it won't be enough, quickly enough, to head off a capacity crunch if current trends continue. The growth rate of smartphones, tablets, and wireless notebooks will swamp the cellular infrastructure no matter what. Folks in the mobile industry have taken to calling this the "Moore's Law vs. Shannon's Law" problem, with Moore's Law representing the exponential growth of computing power, and Shannon's Law the fundamental limits on how much data you can push over a particular chunk of spectrum. Reinforcing the Shannon bandwidth limits is the fact that some other critical elements in the mobile data infrastructure can't keep up with Moore's Law. The number of cell sites can't increase exponentially, and handset battery capacity is barely growing at all. A crunch is inevitable at some point.

So the people who tell you that cellular wireless will replace wired broadband just don't understand the physics involved. An outstanding summary of the situation was written by Martyn Roetter, a telecom consultant (link).

Here's the key paragraph:

"Until and unless the current laws of physics are invalidated in ways that remove current limits on spectrum capacity such as are embodied in Shannon’s Law, the future will see: (a) The vast majority of broadband traffic (as distinct from numbers of broadband subscriptions) continuing to be carried (delivered and transmitted) over fixed access networks; and (b) Demands for broadband traffic from wireless or mobile subscribers outstrip the capacity of all the bandwidth available for radio access networks to handle it, even with the use of the new spectrum that can be allocated and the deployment of more spectrally efficient technologies... Bandwidth within one optical fiber is vastly greater than all the bandwidth that might theoretically be made available for mobile communications, even if every megahertz were to be refarmed for mobile services. A single mode fiber has a bandwidth of as much as 100,000 GHz, or 100 terahertz, whereas total valuable spectrum for mobile communications provides bandwidth of no more than at most 3 GHz."

Got it? What he's saying is that wired broadband traffic can continue to grow exponentially, which will create demand for mobilizing that traffic through cellular wireless -- which the cellular networks can't handle.

If data traffic continues to grow at its current pace, we're headed for a situation in which the cellular networks will be overloaded no matter what we do.


Rock, meet hard place

So we have two possible scenarios for the future of mobile data. In the segmented scenario I discussed in part 1, we run out of customers willing to pay for mobile data plans, and the growth of mobile data slows down. In the consensus scenario, customer demand continues to increase, and we run out of cellular network capacity.

These conflicting scenarios are terrifying to the mobile operators because there's no way to tell for sure which one will happen. If you knew for sure that demand was going to continue to grow, you'd invest heavily in capacity, and also start raising data prices to restrain the growth in demand to something you can actually deliver. But if demand is about to stop growing, investing in capacity and raising prices is exactly the wrong thing to do. You'll end up with excess capacity, and the price hikes will make demand stop growing even faster.

This table summarizes the dilemma (click on it to see a larger version):



An economist would tell you that this will all sort itself out in the long term, and I'm sure it will in 20 years or so. But in the meantime, in the real world, the operators have to invest in infrastructure years before the demand arrives. If you're an executive at a major operator, it is almost impossible to get the forecast right. That means you will probably either overbuild the network, wasting billions of dollars and putting your career at risk; or you will underbuild, losing share to competitors and putting your career at risk.

You can't win. It's like one of those Star Trek episodes where Captain Kirk destroys the rogue computer by putting it in a logical loop (link). If you watch closely at tech conferences, you can see the smoke seeping out of the ears of telecom execs.

Faced with this dilemma, those telecom execs naturally are trying to find a third option: Get someone else to pay for mobile data. There are a couple of options:


Option 1: Have the government pay for mobile data

I doubt that most governments would pay to make wireless data completely free for everyone, but I was surprised when I found out how much governments are already paying for mobile. For example, the US government subsidizes mobile phone service for millions of unemployed people (because it helps with their job searches; they need phone numbers so employers can all to offer them jobs). I could easily imagine that benefit being extended to include mobile data, on the assumption that poor people need access to job boards (how we'll avoid paying for their YouTube and Kongregate usage I don't know).

Governments are also being lobbied to give special regulatory treatment to wireless data. The rhetoric around a "wireless spectrum crisis" is being used to influence governments. The focus of this lobbying in the US is on taking spectrum away from the TV networks and supplying it to the mobile operators. Effectively that is a financial subsidy for the operators -- if the government forces the transfer the operators will have to pay less, and will get the spectrum faster, than if they were to purchase it on the open market.

Here's how the lobbying works. This is an excerpt from an e-mail sent to me recently by a PR firm working for a group called the Internet Innovation Alliance:

"IIA's Blog: The Spectrum Clock is Ticking
Writing for Forbes, Lawrence J. Spiwak, President of the Phoenix Center for Advanced Legal and Economic Public Policy Studies, warns Congress that more spectrum needs to be freed up for mobile broadband and it needs to be freed up soon: Like it or not, the clock is ticking on spectrum exhaustion, both for consumers and our public safety professionals. Unless we want a market characterized by higher prices, failed data sessions, dropped calls and stifled innovation, policymakers need to implement a cohesive spectrum policy with a large degree of urgency."

At first glance, that reads like a standard plea from a bunch of web companies worried about the mobile network getting overloaded. But the backstory is that both IIA and the Phoenix Center are reportedly funded by the mobile operators (link, link). So this isn't an independent assessment of the situation, it's the operators sending us a message. And the message is: "Give us more bandwidth or we'll trash your phone service." I think that's a bit disingenuous -- unless the operators seriously mismanage their networks, we won't end up with both bad service and higher prices. But they're right that without more spectrum we'll definitely get one or the other.


Option 2: Make web companies pay for mobile data

Several of the leading operators in Europe recently argued that big tech companies like Apple and Google should be forced to pay to use the wireless networks. Although they don't put it this way, they're asking the big Internet companies to subsidize mobile data plans for users (link).

The CEO of Telefonica said the web companies "use Telefonica’s networks for free, which is good news for them and a tragedy for us. That can’t continue."

Here's the CEO of France Telecom (link):

"The real risk of everything is collapse. Nobody utters this loudly enough, but the real issue for the world is a collapse of the network or some local collapses. We are the people with pipes. We are supposed to invest heavily in pipes in order to bring the capacity which is necessary to sustain the explosion of consumption and usage and data traffic in our networks. At the same time, the people that create this traffic…are not really incentivized to manage properly, globally, the traffic. There is an unbalance in the overall system, which in our view is a major problem. It is totally impossible to absorb such an explosion in traffic without first, clearly investing massively in spectrum and equipment, and second, without introducing some new pricing approaches."

This is the heart of the whole debate about net neutrality. I believe it's not really about mobile operators trying to give an advantage to their own services, it's mostly about the operators trying to open up a second revenue stream because they're afraid they can't get enough revenue from users to support future growth.

For the operators, charging web companies a fee seems intensely attractive because the fee could be scaled to the amount of traffic they generate (unlike the flat-rate data plans that users prefer), forcing the web companies to use bandwidth more efficiently. It also would let operators increase their revenue without directly reducing user demand. Basically, the web companies would subsidize a shift from wired to wireless computing.


The third option

I can see why the operators are pushing on both of these options. They're in a difficult situation, and it would be very helpful to them if somebody bailed them out (link). I might be trying the same things if I worked for an operator. But there is a third option for managing cellular data overload, and it deserves to get a lot more attention:

Raise prices.

In almost every other industry in the world, you're responsible for charging enough money to support your business. Yes, sometimes you have to make investments before you know how much demand there will be, and yes, sometimes that creates a lot of risk for your company. But that's why they pay you the big bucks, Mr. or Ms. CEO.

I don't understand how we as a society came to the conclusion that wireless data should be different. Is there some religious commandment that people must be allowed to stream Netflix on the subway? Or maybe those big Cisco growth forecasts have led us to think that endless growth of mobile data is a ravenous beast that will cause immense suffering if it's not fed more bandwidth.

Baloney. If the network is overloaded, raise your prices until you either get enough money to expand the network, or you force people to use less data. If you want network bandwidth used more efficiently, show users the cost of the data they use and they'll demand more efficient apps and devices on their own.

I bet a price increase from $50 a month to $80 a month for mobile data would end the bandwidth crisis overnight.

Not only is higher pricing the simplest way to manage network overload, it's going to happen no matter what we do. Even if we give the operators all the bandwidth from the TV networks, and get the web companies to subsidize wireless service, all that will do is delay the crunch for a few years. More traffic will switch from fixed-line to wireless until once again the network saturates and prices go up. It is inevitable.


What it means

When we plan for the future of mobile, we need to be realistic, and a little bit humble, about what we can change and what we can't. We need to learn to live with the things we can't change, and focus on doing a good job of managing the things we can.

Here's my list of the things I think we can't change about mobile data because they are driven by economics and physics:

--Most data traffic will be wireless only for the last 100 feet (30 meters) that it travels from your device to the nearest hotspot (whether it's WiFi, femtocell, or something else). So we need to be careful about our terminology. Most data could well be technically "wireless" in the sense that it passes through WiFi at the end, but that is a meaningless distinction for the purposes of this article; most of it won't pass through the cellular data network.
--Most of us will continue to have some sort of broadband cable connecting to our homes and offices, or to a point very close by (like the lamp post in the street outside your window). Forget those visions of cellular replacing the wired broadband network; in the developed world it can't happen.
--The cost per-byte of cellular data will be significantly higher than the cost per-byte of wired data. The difference will be large enough that we'll be aware of it and it will alter the way we use our devices.
--Flat rate unlimited cellular data contracts will go up in cost, or will be replaced by much more variable pricing for most users. This is already underway at some operators. For example, Verizon is rumored to be about to move from $30 per month for unlimited data to a tiered plan that ranges from $30 per month for two gigabytes to $80/month for 10 gigs (link). I don't usually like consumer price hikes, but in this case the change is long overdue.
--As the relative cost of mobile data rises, most of us will use cellular data primarily as a supplement to the wired network when we're on the go. We'll become religious about turning on WiFi in our smartphones and tablets, and making sure it can connect at home and at work. Because cellular data is more expensive, many of us will try to avoid using very data-heavy apps on the cellular network.

This means cellular data use won't be carefree. That may not sound like a big difference, but in consumer terms I think it is. We've been making the assumption that cellular data can directly replace fixed-line data, just as cellular phones replaced fixed line phones for many people. "Go ahead! Use it anywhere! Be free!" But for an aggressive user of mobile data, that can't happen. Our use of cellular data is going to be much more nuanced, managed, and carefully thought out than our use of cellular voice. I think many of us will look at our cellular data budgets the same way we look at our automobile budgets. Some people will spend more, some less, but I think most of us will be aware of the cost and manage it actively.

The wired Internet will continue to set the tune. The ongoing role of fixed-line broadband means that many leading-edge web apps will continue to be designed around the capacity and responsiveness of fixed-line networks. This is another subtle but very important difference, because it means the mobile operators will continue to play catch-up to customer expectations set on the wired networks. There will be exceptions; some features of cellular data (such as location) will drive unique mobile apps. But in most application categories, rather than shaping the future of the Internet, mobile operators in the developed world will be pushed to deliver an Internet experience that evolved on fixed-line networks.


Here are the things I think we can change about cellular data:

--We can alter the share of total data traffic that moves through the cellular networks. By transferring spectrum and giving the operators other favorable treatment, we can make the overall capacity of the cellular data networks higher than it would have been otherwise. Basically, we can make the mobile operators bigger. That may delay the onset of mobile network congestion, and enable some classes of web applications to be more successful in cellular (for example, low-res video streaming). That can have a big impact on individual users and app companies. It will also have a big impact on the ultimate revenue and profitability of the mobile operators, which is why they are lobbying so hard.
--We can probably change the size of the average mobile data bill, but only temporarily. The more revenue streams we give to the operators, the more mobile data we'll probably get for a given user price. However, as I mentioned above, keep in mind that if mobile data is made cheaper, people will use more of it, which will eventually saturate the network and cause prices to rise. So any money we save on our mobile data bills will probably be temporary.
--The decisions we make in the next few years will profoundly change the economic structure of the wireless data industry. Changes in regulations and pricing rules will have a huge impact on the ability of small companies to compete with large ones in mobile, and will determine who pays for the whole thing. This could decide whether the mobile internet looks more like the wired Internet (low barriers to entry, lots of companies) or cable television (high barriers to entry, dominated by a few big players).

I think the most important thing about the three points above is that they're all driven by government regulation. The rules we set for the mobile Internet are going to determine the ultimate size of the mobile operators, how they are funded, how competition works in mobile data, and how much power is held by the various players.

That scares me. I prefer to have winners and losers in a market chosen by customer decisions, not government ones. You can't blame the mobile operators, or the big web companies like Google, for lobbying the government on these issues. But I don't think their interests are necessarily the same as the rest of the industry, let alone consumers. Also, most of the big players are driven by quarterly revenue, and in some cases they are pushing for changes that I think will help them in the short term but would actually hurt them in the long run.

I wish there were some scenario in which we could tell governments just to butt out and let the market decide, but governments are already deeply involved in allocating spectrum, and there's no practical way to undo that. So I think it's important that we all have a very thorough, open discussion of the government decisions to be made and the sort of wireless industry they'll produce.

That's what I'll cover tomorrow.

_____

In part 3, which I'll post tomorrow (link), I'll give my ideas on how we should structure the mobile data market. I'll also talk about the opportunities this new world of mobile data will create for companies in mobile. In the meantime, I welcome your comments and questions. This is a big, complex issue, and I don't pretend to have it all figured out.

Who Will Pay for Mobile Data?

(Part 1: The End is Nearer Than You Think)

The most important question in mobile computing is who's going to pay for all that mobile data we're supposed to use in the next few years.  The question doesn't get much discussion online, but it's at the heart of the most intense debates in mobile, including net neutrality and the wireless bandwidth "crisis."

How many users will pay for their own mobile data service?  Will web companies also pay?  Will the government step in?  And most important, how much money are any of them willing to pay?  The answers will shape the future of every company involved in mobile, and will have a profound effect on everyone who uses a mobile phone.

Here's a quick summary of my answers:
     --Because of economics and user psychology, I think we're headed for a slowdown in the growth of mobile data.  The unlimited, exponential growth forecasts are wrong.  I believe the ultimate mobile data market will be smaller, and much more segmented, than most people expect.
     --Even if I'm wrong about user demand, we're still headed for a slowdown in growth because the cellular networks can't grow fast enough to handle all the traffic being forecasted.  This is due to physics and can't be changed; you could just as easily change the phases of the moon. 
     --Many of the proposals to "fix" the problem would probably make it far, far worse.  We could end up with a cellular data network that resembles American cable TV: slow to innovate, dominated by a few players, and subject to intense politics, with users caught in the middle.
     --In the future, cellular data for the majority of users will likely be metered, and the majority of people will need to be enticed into using it.  That creates some excellent unaddressed opportunities for everyone from handset companies to app developers. 

This is a very complicated issue, so I'm going to cover it in three posts this week:

Today's post talks about the forecasted growth of wireless data, and why I think growth won't continue the way most people are expecting.  That creates some big challenges for mobile data companies, but also some fantastic opportunities.

Tomorrow I'll talk about the alternate scenario, in which mobile data growth continues at the same rate, eventually colliding with natural limits on the amount of data that can travel over a cellular network.  I'll discuss how that collision drives the rhetoric about a bandwidth "crisis" and the debate about net neutrality.

In the third part, I'll discuss what it all means for the industry, and give my take on what we should do about it.

To start today's post, let's look at the predicted growth of mobile data...


The forecasts for mobile data growth are so sunny they could burn your skin

Every mobile phone will be a smartphone.  Smartphones are already used by about a third of the US mobile population (link, link), and ownership rates are similar in parts of Europe (link).  Horace Dediu says half of US phone users will be on smartphones by the end of 2011 (link), and non-smarthones will be virtually extinct a year later (link). 

Mobile data traffic will explode.  Cisco says global mobile data traffic will increase 26X from 2010 to 2015 (link).  The growth will be driven by increased use of smartphones, and also a rise in the number of notebook PCs connected to the cellular networks.  Notebook computers generate an order of magnitude more data traffic than smartphones, so even a small number of cellular notebooks drives a huge increase in traffic.

Mobile app shipments are off the chart.  Apple says iOS users download 62 apps per device on average, for a total download rate of 206 apps every second (link). 

The big tech companies are focused on mobile.  Many of the hottest tech companies, most recently Facebook, say their biggest focus for this year and beyond is nailing the mobile opportunity (link). 

PCs will be replaced by smartphones.  A Google vice president says smartphones will render PCs irrelevant by 2013 (link).

The growing consensus is that our current cabled, PC-centric computing world will soon be replaced by an untethered world in which everyone uses smartphones and tablets to do their computing on the go.  The mobile network we all envision will be just as flexible and carefree to use as today's wired Internet, but with the added benefit that you can use it anywhere, anytime, with a wide variety of different devices. 

It sounds cool, but unfortunately no one has ever asked users if we're all willing to pay for that mobile data service.  I think most of us aren't.


We won't pay for all the mobile data we want

We'll all carry smartphones, but...  The forecast for mobile data growth is based in part on the assumption that in the near future most or all mobile phones will be smartphones.  You can make a good case for that assumption.  The price of smartphone components is continually decreasing, so at some point the parts cost a smartphone will be the same as a feature phone is today.  Even if prices aren't completely equal, once they get close, it's more cost efficient for a mobile phone company to base its phones on a smartphone OS because it requires less rewriting of apps and support software for each new phone.  The handset companies have an incentive to switch to smartphone hardware.

So I have no doubt that in the next couple of years most phones sold in the developed world will technically be smartphones.  However, I think it's not reasonable to assume that they'll all be used as smartphones, because many users won't be willing to pay the data charges.

As I've written before (link), when I was at Palm we did a lot of research on mobile phone users in the US and Europe, and we found that about a third of them were willing to pay extra for new mobile data features in addition to voice and texting.  Some of them were more interested in entertainment, some in business communication, and some in information management.  These are the people who have been buying iPhones and BlackBerries. 

The other two thirds of phone users were not willing to pay extra for any new sort of mobile data.  Some of them didn't have enough income, some of them just weren't interested, but they all flat-out refused to consider spending extra.  The Palm surveys were conducted several years ago, but since then I have seen no evidence to suggest the basic situation has changed.  On the contrary, the most recent research I've seen was done by Forrester in 2009, and it suggested the unwilling-to-pay share of the population may have dropped from 66% to about 60%.  So there is movement toward more willingness to pay, but it's very gradual.

It's hard for me to believe that most of those 60% will be willing to add about $400 a year to their mobile phone bills just for the privilege of checking their e-mail on the bus or streaming songs from Pandora onto their phones.  And remember, that research data is based on people in some of the richest countries in the world.  It may map fairly well to other rich countries like Japan and South Korea.  But in the developing world, average personal incomes can't possibly support big mobile data bills.  Most people there will need to sip data through a straw rather than gulping it from a mug.

So I have a fundamental disagreement with many industry analysts about how the mobile data market will develop.  A graphic would help explain...



This has a huge impact on what will happen next.  The consensus view says that with only a third of the population in the US and Europe owning smartphones today, the prospects for growth are fantastic -- we can still sell to the other two thirds!  And after that we'll move on to the rest of the world.  The segmented view says that with a third of the population using smartphones in the US and Europe, we've already sold to most of the world's population willing to pay for big data plans.  In this view, data plan growth will start to slow in the US and Europe by sometime in 2012.

The best way to check which scenario is right would be to conduct some market research on user willingness to pay for data plans.  If you work in a mobile tech company, you should be doing that, urgently.  For those of us without six-figure market research budgets, there are some warning signs to look for.  If the segmented view is correct, we should start seeing more price sensitivity as we use up the late adopters of data plans.  One sign would be price promotions on smartphones...


AT&T's most recent iPhone advertising (link).


Another sign would be a shift in the mix toward lower-cost data plans...


Growth in data plans, 2010 vs. 2009.  In all five countries, growth is higher in mid to low-tier plans (under 50 euros / 35 pounds a month).  Source: Comscore (link)

This isn't conclusive evidence, but you don't get conclusive evidence until something has already happened.  There's enough evidence that we should be talking very seriously about possible saturation of the user segment willing to pay for mobile data.


What it means

So to recap, in a few years I think the majority of phone users will have smartphones but won't necessarily pay for today's data plans.  Some of the phones will connect to the web by WiFi only, while others will be on pay-as-you-go plans and won't be used for much data at all.  The situation is analogous to what happened with cameraphones.  Almost all of us have cameras in our phones, but most of us don't send picture messages because of the cost. 

This stratification of data use will have some pretty profound impacts on the mobile market:

A change in the crisis.  The first effect will be that we'd hear a lot less about the wireless bandwidth "crisis."  Operators will all of a sudden feel a lot less pressure to expand their networks and get more spectrum.  However, they will not be happy.  Slowing data growth will probably make them miss their revenue forecasts, hurting their stock prices.  Some operators may end up with excess capacity, resulting in renewed price competition in data plans, and putting more pressure on earnings.  So instead of a bandwidth crisis we'll suddenly have an overcapacity crisis.

Pressure on mobile startups.  Right now mobile apps are seen as a hot investment area because there's so much growth.  There's a lot of venture capital available.  If growth of mobile data slows, the rate of investment will slow also, as investors look for the next hot thing.  This won't be a disaster for today's mobile app companies, but it would make life harder for new entrants.  Also, companies that are investing on the assumption of endless growth might find themselves overextended.

Data will go a la carte.  But the biggest change is that to make mobile data grow further, we'll need to entice people into using it.  The challenge will be getting them to pay for little bits of data service, one app or one occasion at a time.  This requires a different sort of data plan, different apps, and a different user experience on the phone...


Enticement becomes job one

A mobile data slowdown will create an enormous opportunity for smartphone companies and app developers to create a different sort of relationship with phone users.  Most users will be perfectly willing to use data; they just won't want to pay for the plans.  The single most important task for driving mobile data growth will be to gradually entice these people into using data a bit at a time.  This creates several big business opportunities:

"Toll free" applications.  Just as we enable toll-free phone numbers in which the recipient of the call pays, we should enable toll-free apps and websites in which the app or site vendor pays for the cellular data charge.  I can picture several uses for this:
     --Some sites or apps might be willing to pay the data charge because they earn enough from ads to cover the cost.  For example, I am willing to bet that Google and Bing would both pay the data charges for a mobile search on their sites. 
     --Some sites or apps might be mobile supplements to paid PC web apps whose monthly service fee is large enough to cover the mobile data cost.  This might apply to a music streaming service or a file storage service.
     --Some third parties might be willing to cover the service fees for an app or website.  For example, the movie Rio sponsored a version of Angry Birds.  Picture them doing the same thing with a web app that transfers data.  They don't want the users hesitating to use their app, so they will pay the data charges.

Although it's easy to talk in the abstract about toll-free data apps and websites, it will be hard to implement them.  We'll need a payment clearinghouse that standardizes and manages the transfer payments between developers and mobile operators.  That was done for toll-free numbers, so I assume it should be straightforward, but there's still a lot of work to do. 

We'll also need a way to let the users know about toll-free apps and websites.  I think this is a task for the operating system -- it should identify the toll-free apps and sites automatically and enable them on phones that don't have data plans.  The operators also have work to do, because they'll need to track the data used by the toll-free apps and make sure it's not charged to the user.

It might also be good to have a top-level web domain for toll-free mobile sites.  I think .up (for "unpaid") is available.

There is also an important role for government here: Don't screw this up.  We need to be sure that any net neutrality regulations don't accidentally ban toll-free sites and apps.  It's possible that toll-free apps and sites will end up being the main way most people access mobile data, and it's critical not to cut off that possibility.  (I'll discuss net neutrality in a lot more detail in the second and third parts of this post.)

After-sales billing is critical.  Mobile and web developers have already figured this out: In many cases, your best chance of making money is to give away your base product and charge for upgrades and add-ons.  That business model becomes even more important in a world where most users don't have a mobile data plan.  How do you gradually get people hooked on your product when they're not willing to even pay for the cost of connecting to your website?

For some developers the answer will be that you just ignore those customers (and in that case you'd better base your forecast on selling to only a third of the population).  But for other developers, there will be an art in figuring out how to write a very data-efficient app or website that delivers enough value to hook a user with a data charge so low that you can pay it, at least during a trial period.  That sort of art is a great opportunity for differentiation.

Micropayment is critical as well.  Because developers need to experiment in incremental billing, it's critically important that they be able to easily bill customers in very small amounts.  The best system for doing that looks to be Google's recently-announced In-App Payments system, which is supposed to launch this summer.  Google will charge a flat 5% of your revenue no matter how small the transaction.  This is a huge improvement over PayPal and Amazon FPS, both of which charge 5% plus 5 cents per transaction (in other words, they take 40% of a 25-cent transaction). 

If the operators want to facilitate this sort of billing through their own infrastructure, they'll need to match Google's terms.  Operators that are wise enough to enable this may be able to build tight alliances with the most innovative websites and apps, but my guess is that most operators won't be able to get comfortable with a cut as small as 5%.  In that case, they should just get out of the way and let Google (and its competitors) operate.

Smartphones must entice.  This is a huge opportunity for companies that make handsets and mobile operating systems.  Smartphones today are designed for unlimited data plans -- here's the browser, click away; here's the app store, download something.  Those apps will be ignored by a user who has a limited data plan.  Instead, the phone itself will need to show the user individual functions and apps they can use for small bits of money.  Want directions?  That'll cost you 25 cents.  Want to download an ebook?  That's a buck.  Folks in Europe already understand this sort of world well, because so many users there are on pay-as-you-go plans.  But to most Americans it's a new concept.  Get used to it.  Think of mobile data like an a la carte menu in a restaurant, except that for data the options are almost infinite -- so the phone will need to learn about the user and customize the offers to his or her particular interests.

This model of infinite customization and a la carte ordering requires a fundamental redesign of the user experience of the smartphone.  That means it is a huge opportunity for differentiation, maybe the biggest single opportunity in mobile computing.  Apple is the leading vendor in smartphones for people with large data plans.  Although Android is catching up on many countries, often it seems to be selling to the more price-sensitive end of the market (note the lower sales of paid apps on Android compared to iPhone).   So it makes sense that the "enticement phone" would be built on Android.  I'd like to think Google would do it, but intuitive and well-integrated user experience is not its strong suit.  So maybe it'll be an Android vendor.  Or maybe Nokia will do it.  Or even Microsoft.  Whoever gets it right first has a very good chance to be the other dominant smartphone vendor.

Or maybe Apple will do it first, and end up the leader in all smartphone price bands.  It wouldn't surprise me.


What if I'm wrong?

So that's what I think is going to happen: there will be a natural slowdown in the growth of mobile data as we use up the customers willing to pay for it, and the most critical task for mobile data companies will be enticing people to use their services a bit at a time.  But what if I'm wrong?  What if the whole population is so excited about smartphones that everyone is willing to pay for big mobile data plans?  How does the world look then?

I'll cover that tomorrow, in part 2 (link).  In the meantime, please post comments and questions.  This is a huge, complex issue, and I don't pretend to have it all figured out.

Browsing the web for $10,000 an hour

Over the years I've seen those stories about travelers who make the mistake of using mobile data services outside of their home countries and end up with ridiculously big phone bills. But I didn't realize how easily it could happen until my colleague, Nilofer Merchant (link), got her phone bill last week.

Nilofer recently made a business trip through Canada. While waiting in the Toronto airport for her flight home, she fired up her laptop. The WiFi service in the airport costs about $10 a day, so she decided to use her AT&T data card instead.

Big mistake. Big, big, big mistake.

When her phone bill arrived last week, the total charge for that airport session was $10,609.27.

Needless to say, this was a topic of pretty intense discussion at the office last week.

The stories I'd heard about people with big roaming bills were usually about someone on vacation who used data for a couple of weeks. I was very surprised to find that you could run up such a huge bill in just a few hours. Here's how it happens:

AT&T's Data Connect plan gives five gigabytes of data transfer a month, with additional data priced at five cents per megabyte. But when you're roaming in Canada, there is no prepaid data allowance, and the charge is $15 per megabyte -- 300 times higher than the charge at home.

Nilofer supposedly transferred 707 megabytes in that airport session, which adds up to over $10,000.

I'm not sure how she could have generated that much traffic in a few hours. She said she was doing normal business tasks, not watching videos. It's barely theoretically possible for someone to use that much data in the time available. PC World reports that AT&T's network can transfer about 1,400 kilobits per second (link), which means 707 megabytes could theoretically be transferred in a little bit over an hour. But that assumes a single continuous connection, running at full speed for the entire time. You're not likely to get that in a real-world browsing session, which is full of starts and stops.

You'd think AT&T would warn a customer when they're building up this sort of charge, but that's not the case. The only notification was two form e-mails AT&T sent after the data session was already over. The first said her data service had been shut off due to excessive charges, and the second --dated one minute after the first one -- warned of high usage. That message claimed that "AT&T has sent your end user multiple text messages regarding their high level of international data usage." No text messages were ever received, and in fact I don't know how you would send text messages to a data card.

But the biggest question is not how much data was transferred, or why AT&T doesn't notify customers properly; it's why the roaming charges are so high in the first place. There are a lot of excuses given for that by the operators, but what it comes down to is a cooperative effort between the operators to fleece each-others' customers when they roam. When AT&T customers roam to Canada, they pay 300 times the home rate for data. Meanwhile, when Canadians on the Rogers network roam to the US, they pay 200 times the home rate for data -- unless they have Rogers new One Rate plan, which eliminates roaming charges in the US (link).

The interesting thing about the new Rogers plan, which was introduced last month, is that it proves it's possible to create reasonable roaming charges throughout North America. The operators just choose not to. Because they make a lot of money from it.

The data roaming charges, and their impact on your bill, are not completely hidden by the operators, but they come pretty close. The information is scattered in several locations, and little or no effort is made to explain what the charges mean in practical terms. To find the charge at Rogers, you have to look on their website here, click on the "Legal Disclaimer" link in the tiny type at the bottom of the page, and scroll down to footnote 4. Unless you're technical enough to understand the difference between MB and kb, you may not even realize that roaming costs extra.

AT&T's site is almost as obscure. The company's page with tips on international roaming is here. It discloses the charges for roaming, but doesn't explain how those charges compare to home-country charges. As is the case with Rogers, you're expected to spot the KB vs. MB distinction, and know what it means.

AT&T also provides a helpful map showing its coverage in the US and Canada. Nowhere does it warn of the roaming charges for data in Canada.



The closest thing I could find to a warning about charges was another window with "laptop travel tips" that contains this message: "Your LaptopConnect service provides access to email, Web browsing, and VPN applications that can use a significant amount of data, so remember -international data roaming can get expensive quickly."

A more honest notification might be printed in big red letters, and would say something like this:

"Using wireless data outside your home country is about as smart as juggling chainsaws. In a single day, you can build up charges large enough to buy us a new car. We're constantly amazed that people keep falling for the roaming thing, but you know what PT Barnum supposedly said about suckers. If after reading this you're still stupid enough to use roaming data, please stop by our headquarters the next time you're in Dallas and we'll buy you a drink (although knowing you, we can probably stick you with the tab for that as well.)"

There's a continuous buzz online from people who have been caught by the roaming trap.

Adam Savage, co-host of the television show Mythbusters, had a similar incident last year (link). After he raised a stink on Twitter, the charges were dropped.

You can read some more examples here and here and here and here and here. It makes you wonder why some politician hasn't taken up this issue. A nice round of Congressional hearings would be fun (I'm looking at you, Nancy Pelosi).

In response to complaints in Europe, the EU recently regulated roaming fees and capped roaming data charges at one Euro per megabyte, about a tenth of AT&T's charge (link). As a rugged, individualistic American, I'm generally skeptical of the EU's reliance on the dead hand of regulation. But in this case, I congratulate my friends in Europe, and I say bring on the bureaucrats.

The mobile data apocalypse, and what it means to you

The mobile industry is now completing a huge shift in its attitude toward mobile data. Until pretty recently, the prevailing attitude among mobile operators was that data was a disappointment. It had been hyped for a decade, and although there were some successes, it had never lived up to the huge growth expectations that were set at the start of the decade. Most operators viewed it as a nice incremental add-on rather than the driver of their businesses.

But in the last year or so, the attitude has shifted dramatically from "no one is using mobile data" to "oh my God, there's so much demand for mobile data that it'll destroy the network." A lot of this attitude shift was caused by the iPhone, which has indeed overloaded some mobile networks. But there's also a general uptick in data usage from various sources, and the rate of growth seems to be accelerating.

Extrapolating the trend, most telecom analyst firms are now producing mobile data traffic forecasts that look something like this:




The forecasts are driven by a couple of simple observations:

--Smartphones produce much more data traffic than traditional mobile phones. Cisco estimates that a single smartphone produces as much data traffic as 40 traditional feature phones. So converting 10 million people from feature phones to smartphones is like adding 390 million new feature phone users, in terms of impact on the data network. The more popular smartphones get, the busier the network becomes.

--A notebook PC generates far more traffic than a smartphone. According to Cicso, a single notebook computer generates the same data traffic as 450 feature phones. As notebook users convert to 3G-enabled netbooks and add 3G dongles to their computers, they dramatically increase the data traffic load on the network.

You can read Cisco's analysis here.

This becomes especially interesting when you look at the forecasts for growth of 3G-equipped netbooks and notebooks. Mobile operators in many countries have started subsidizing sales of those devices if you pay for a data service plan. It's an attractive deal for many people. Say your son or daughter is going off to college. Do you buy them a regular notebook computer and also pay for the DSL service to their apartment, or do you buy them a 3G data plan for about the same price as DSL and get the netbook for free?

The forecasting firm In-Stat recently predicted that by 2013, 30% of all notebook computers will be sold through mobile operators and bundled with 3G data plans (link). Notebook computer sales worldwide are about 150 million units a year, so that's 45 million new 3G notebooks a year -- or the data equivalent of adding 20 billion more feature phones to the network every year.

Jeepers.

These forecasts are producing a behind-the-scenes panic among mobile network operators. The consensus is that there's no way their networks can grow quickly enough to support all that data traffic. There are several reasons:

--They can't afford to build that much infrastructure.

--Even if they could afford the buildout, they won't have enough bandwidth available to carry all that data, even with 4G.

--Traffic-shaping techniques like tiered pricing and usage caps can't restrain usage growth enough to save them, because

--Fear of losing customers to a competitor will force them to continue to subsidize sales of 3G dongles and offer relatively generous caps in their data plans.

There are a number of projections that show the operators losing money on wireless data a few years from now, as costs continue to increase faster than revenue. The danger isn't so much that they will all go broke, but they're very afraid that they'll turn into zero-profit utilities.

Many operators now seem to be counting on WiFi as their ultimate savior. The theory is that if they can offload enough of the data traffic from their networks to WiFi base stations connected to wired networks, then maybe other measures like 4G, usage caps, and aggressive improvements to the network will let them squeak through.

It's an ironic situation. For a long time the mobile operators thought of themselves as the future lords of data communication. All devices would have 3G connections, the thinking went, and the fixed-line data carriers such as Comcast and BT would fade away just like the fixed-line voice companies are doing.

Instead, the new consensus is that we're moving to a world where the fixed-line vendors will be expected to carry most consumer data traffic for the foreseeable future. They'll provide your wireless connectivity at home and work, while the mobile network will fill in the gaps when you're on the move. The area of disagreement, of course, is who will get the majority of the access revenue. We'll let the fixed-line and mobile operators argue over that one; I want to talk about some of the other impacts of this weird new hybrid wireless world that we're heading into.

(I touched on some of this in my post on net neutrality a couple of weeks ago (link), but I want to go into more detail here.)


The brave new world of scarce mobile bandwidth

Built-in WiFi is now good. For a long time many mobile operators resisted selling smartphones with WiFi built in. They viewed WiFi networks as competitors for customer control, and wanted to prevent usage of them. Now that they see WiFi as their savior, the operators are suddenly encouraging its inclusion in phones. Don't be surprised if in the near future it becomes impossible to get a subsidized price for any smartphone that doesn't have WiFi built in.


Traffic shaping is a fact of life, and a likely source of irritation. Many mobile operators are starting to limit the performance of applications that consume the most data bandwidth (today that's mostly video and file sharing). It's already being done today, and in most cases the operators won't even tell you they're doing it, unless the government requires them to. Certain apps will just communicate more slowly, or fail altogether, when the network gets busy.

There are a couple of exceptions where operators have been more public about their traffic shaping activity. The 3 network in the UK recently announced restrictions (link). And O2 in the UK has given details on exactly which applications it restricts in its home wireless data service (link).

Current traffic shaping hasn't generated a firestorm of complaints from the average customer (as distinct from net neutrality advocates), in part because it is very hard for users to tell why a website runs slowly on a particular day. But as mobile traffic continues to increase, operators are going to find that it's cheaper to ratchet up the restrictions bit by bit rather than pay for more capacity. Eventually people will notice, and I worry that we'll end up in a situation in which the operators carefully balance out how much they can piss off their customers without creating an outright revolt. It's a lot like the way the US airline industry operates today, and it's a miserable experience for everyone involved.

What to do. There are better ways to shape traffic. I think operators should give customers more information on how much data they're using at any given time, so they can manage it themselves. Then let them make an informed decision about which apps they'll use their bandwidth on. It would be relatively simple to create an on-screen widget showing how much data is being transferred at any time, just like the signal strength and battery life indicators on today's phones.

It's also possible to create some APIs that would tell a website how much bandwidth is available to it, so the developer could adjust its features accordingly. This idea is being tossed around between web companies and operators, but I don't know how much is actually being done about it.

Combine those changes with usage-based pricing (my next point) and customers will shape their own traffic. Then there won't be any need for covert manipulation of the network.


Say hello to capped data plans. Completely unlimited wireless data plans are not sustainable long term; the economics of them just don't work. And in fact, virtually no data plans today are completely uncapped; there is almost always some fine print about the maximum amount of traffic allowed before surcharges kick in or the user is tossed off the network.

Some people are saying that the operators should go back to charging by the byte, and in some parts of the world (particularly Asia), there is a long history of per-byte pricing. But the experience in most of the world has been that per-byte pricing makes users so nervous about their expenses that they won't use data services at all.

(DoCoMo in Japan has an interesting hybrid approach (link) in which it charges per-packet until the user hits a maximum charge of about $70 per month. Additional usage beyond that cap is free. So that's capped pricing rather than capped usage. This reduces customer fear of accidentally running up a gigantic bill, but I wonder how DoCoMo prevents power users from flooding the network with traffic. Maybe there's a second, hidden cap on total usage.)

What to do. I think the right answer in most of the world is going to be flat-rate data plans in which there's a clearly-communicated cap, with tiered charges beyond that. The cap will need to be set at a level that moderate users won't ever reach, so they don't become gun-shy about data. To alleviate the fear of accidentally running up a huge bill, there will also need to be an on-device meter showing how much of the user's monthly data allocation has been used (just telling them to go look at a website is not enough; it should be on-screen). I'm told that on-screen meters like this are already being offered on netbooks by some European operators.

Today most operators are pretty up-front about communicating the data limits when a computer is connected to a mobile network. But many of them are still deceptive toward smartphone customers. AT&T's Smartphone Personal service, for example, promises the following for $35 a month:

Included Data: Unlimited; Additional data: $0 per MB

Sounds pretty straightforward. No asterisks, no fine print. But if you click on the terms of service (link), you'll find a long list of banned application types, followed by this general provision:

"AT&T reserves the right to (i) deny, disconnect, modify and/or terminate Service, without notice, to anyone...whose usage adversely impacts its wireless network or service levels or hinders access to its wireless network... and (ii) otherwise protect its wireless network from harm, compromised capacity or degradation in performance."

In other words, if the network is getting slow, they can do anything to your service, at any time, without notice.

There is also a hidden 5G per month maximum:

"If you are on a data plan that does not include a monthly MB/GB allowance and additional data usage rates, you agree that AT&T has the right to impose additional charges if you use more than 5 GB in a month."

This is not just an American problem. Orange in the UK calls its iPhone data service "unlimited," but there's a footnote saying that "unlimited" actually means 750 megabytes a month, a surprisingly low cap compared to AT&T's.

If we're ever going to collectively manage mobile network overload, we'll all need to be much more up-front about the way it operates and what a particular service plan will and won't do.


Is residential 3G really a good idea? Especially in Europe, it's common for operators to tell people that they should ditch their DSL or cable modem at home and replace it with a 3G modem. That works out well only when the network has excess capacity. As soon as the networks start to get congested, the operators will need to offload traffic to residential WiFi routers connected to DSL or cable. If those residential fixed lines have been removed, the operators can't offload.

What to do. I think this one is going to be self-limiting. Once 3G bandwidth gets scarce, the operators will realize that they can get a lot more revenue feeding data to smartphones than to PCs. The math works like this: With a given amount of bandwidth, you could support a single notebook computer and charge about $50 a month, or support 11 smartphones at $30 a month each. Hmm, $330 a month versus $50, seems like a pretty easy decision.

But there are two circumstances in which it would make sense for the operators to keep subsidizing PC sales:

1. If smartphone sales plateau. If this happens, eventually the network will catch up with demand and then there will be excess capacity for PCs; or

2. If operators can route most of the actual data traffic from PCs through WiFi connected to landlines. In this case they could sell you data plans knowing that you won't affect their networks much. That brings us to the next point...


Operators have a huge vested interest in unlocking WiFi access points. Most WiFi access points today are encrypted and inaccessible to other devices in the area. I think there's a strong financial incentive for mobile operators to work with fixed-line access companies to get those access points unlocked. The benefit for the wireless companies is clear -- the more WiFi points they can talk to, the fewer cell towers they need to build. But the benefits for the fixed-line operators are much less clear. Why should they help the mobile operators with their bandwidth crunch?

What to do. The ideal situation would be a revenue-sharing deal in which the operators share some money with the fixed-line companies to encourage them to open up access to their networks. In this scenario, your DSL or cable provider would give you a WiFi router that has been pre-configured to automatically and securely share excess bandwidth with mobile devices in the area. Your own traffic would get priority, but any extra capacity could be shared automatically. The benefit for you as a consumer would be a free router, and/or a lower DSL bill as the cable company passes along some of the revenue it gets from the mobile operators.

The effectiveness of this sort of approach is going to depend on the relative cost for an operator of subsidizing a set of WiFi base stations in an area, versus the cost of installing more wireless capacity. I wonder about weird scenarios like a DSL provider auctioning off excess WiFi capacity to wireless operators in a particularly congested area.


Femtocells for the rest of us. Another very logical step for the operators is to start pushing femtocells aggressively. (Femtocells are radios that work like a short-range cell tower, but are the size of a WiFi router. You connect one to your DSL or cable line, and it offloads traffic from the wireless network. Link)

What to do. Today femtocells are generally sold as signal boosters in areas with marginal wireless coverage. But in the future I think it may make sense for operators to give away femtocells, or at least subsidize them, for customers who live in areas where the data network is congested.


What it all means: Fixed-mobile convergence with a twist

If you step back from the details, the big picture is that we really need a single integrated data network that encompasses mobile and fixed connections, and switches between them seamlessly. People have been talking about this sort of thing for years (check out the Wikipedia article on fixed-mobile convergence here), but the focus has generally been on handing voice calls between WiFi and cellular. That's hard to do technologically (because you can't interrupt a voice conversation during the handover for more than a fraction of a second). Besides, it doesn't solve a significant customer problem -- the voice network isn't the thing that's overloaded.

The place where we could really, really use fixed-mobile convergence is in data. I'm worried, though, that the intense competition between the wireless and wired worlds will make it difficult and slow to achieve the coordination needed. This might be a useful place for government to put its attention. Not in terms of regulating the integrated network into existence (that would be the kiss of death), but to grease the skids for cooperation between the mobile and fixed-line worlds.


Just one more thing...

Everything above is based on the assumption that those Cisco and analyst forecasts are correct. But Cisco has a vested interest in hyping fear of the data apocalypse (Emergency! Buy more routers now!!), and my general rule about tech analysts is that every time they all agree on something you should bet against them.

There is a genuine crunch in mobile data capacity going on at the moment; you can read about network outages caused by the iPhone even today. And I can assure you that for every network failure you read about, there are dozens of other failures and near-failures that don't get reported. Many wireless data networks are very stressed.

And the situation will get worse.

But there's no such thing as infinite demand. At some point the growth of mobile data will slow down, and it's very important to try to estimate how and when that'll happen, so we as an industry do not overshoot too badly. The question isn't whether the growth forecasts are wrong, it's when they will be wrong.

I'll write about that next week...