In which Jacob and I discuss Tumblr, Facebook, Twitter, copyright, and the challenge (and imperative) of making money for social networks, content creators, and publishers.
In which Jacob and I talk about Google's (and my) experience in China, why foreign tech companies will almost certainly fail there, and why China's government is slowly losing its grip over information.
Librarians + technology = a personal nirvana. There is no more awesome set of people doing more important work than the librarians and their nerd allies at the bleeding edge of library tech -- they are engaged in an underappreciated struggle to work out how mankind is going to preserve, extend, share, and democratize the sum of human knowledge in our increasingly digital age. So I was really psyched to go a do a talk at the 2012 Library Technology Conference about the technological forces driving the great policy issues of our age, along with an argument about why and where the library community should be engaged. Bonus for me: The event was at Macalester College, where I spent my high school summers taking Russian while trying to look like something other than the huge dork I was.
Here's my keynote, "Fight for the Future: Libraries, Tech Policy, and the Fate of Human Knowledge."
The Prezi is here.
The Ford Foundation has posted the video of a panel I recently joined on "[t]he possibilities and pitfalls of technology in the pursuit of human freedom." The other panelists were (the legendary and eloquent) Sir Tim Berners-Lee, (the brilliant and soon-to-be-book-launching) Rebecca MacKinnon, (the worldly and effective) Elisa Massimino, and (the multi-disciplinary and polymathical) Danny O’Brien. The moderator was (the charming and prolific) Sewell Chan, deputy editor for The New York Times. For me, the most interesting part of the panel was an exchange primarily with Elisa about the pressing need for mainstream/mainline human rights advocates and organizations to view seemingly-specialized tech issues like net neutrality, competition policy, intermediary liability, encryption policy, and user data control as core human rights issues.
Let's go to the tape.
Walk down any major street in any city in the world and you’ll pass by hundreds of pedestrians — and, let’s be honest, more than a few drivers — typing into smart phones. Each of these individuals holds in one hand more computing power than the entire NASA space operation that delivered men to the moon and back in 1969.
It can be hard to recognize how fast and how deeply information technology is changing our day-to-day lives, and how profound the implications can be. Every year, thanks to Moore’s Law, the speeds of our laptops and devices go up even as the cost of computing goes down. Internet bandwidth gets ever faster and cheaper, and wireless connectivity ever more ubiquitous. As a result, the cost of creating, organizing, analyzing, and distributing information has plunged dramatically over the past two decades. Today, an ordinary individual can communicate instantly with any connected person anywhere in the world; she can broadcast her ideas globally, readable by anyone with an interest.
Thanks to the ever-improving economics of computing, a company like Facebook can offer hundreds of millions of individuals the ability to tap a vast computing infrastructure optimized to publish their writings, deliver their messages, and store vast and growing oceans of multi-megapixel images — and all for free, funded by tiny incremental payments from advertisers. That’s how efficient and cheap computing power has become.
To put that in context, consider this: It currently costs about $150 to buy a hard disk with a terabyte of data storage, and it sits on the corner of your desk; as recently as the early 1990s, you would have had to spend over $1.5 million to buy the same thing, and it would have taken up a full corner of your room.
Or this: The iPad 2 is roughly 1,500 times faster than the CDC 6600, NASA’s fastest supercomputer in 1969. The cost of an iPad 2 is currently $499; the cost of a CDC 6600 in 1969 was roughly $58,000,000, in inflation-adjusted 2010 dollars.
Amazing, right? But here’s the problem: As William Gibson puts it, “the future is already here — it’s just not evenly distributed.” In particular, our governments — the agencies and departments and legislatures and courts that our democratic processes have ordained to serve the public — are finding it extraordinarily difficult to understand, much less embrace, the possibilities created and imperatives imposed by the technological advances we take for granted in our private lives.
Civic Commons, the new initiative I’m pleased to be joining today, is an effort to answer that problem. We believe that governments — especially the cities, towns, and counties that are on the hook to deliver public services every day — can now take advantage of the same technologies and techniques that have generated such enormous efficiencies and enabled such impressive new services by private enterprise. In a digitally interconnected world, cities don’t have to operate in isolation. They don’t have to reinvent (or re-procure) the wheel every time they face a problem that technology could help address. Cities can pool their resources — their talents and ever-shrinking budgets — to build shared technologies. Just as open standards (e.g., the Internet protocols), shared infrastructures (e.g., cloud computing), and collaborative software (e.g., open source projects like Linux, Mozilla Firefox, and Apache Hadoop) have powered astonishing advances in personal and enterprise computing, it is now time for governments to put them to work for the public good.
We believe that governments can now build and deploy shared technologies — open standards, common infrastructures, collaborative projects, and open source software, together with proprietary systems — to improve public service delivery, transparency, accountability, public participation, and management effectiveness, all while spending less.
In sum, Civic Commons is built around two central convictions: first, that wave after wave of innovation is delivering amazing new capabilities to the people and organizations that can take advantage of them, and second, that, with a little help, governments can absolutely understand and seize the opportunities created by the rapid evolution of information technology.
So how are we going to do it? Civic Commons will operate as a neutral and expert non-profit that (a) helps cities and other governments understand the possibilities and pitfalls around shared technologies, (b) provides technical assistance, (c) facilitates the creation and management of collaborative technology projects, (d) connects interested cities with peers, collaborators, experts, vendors, and other supporters, and (e) creates high-quality information — such as guides, checklists, how-to’s, and a comprehensive catalog of civic technology — that is as comprehensible and useful to mayors, city managers, and citizens as it is to software engineers.
We’ve already got a great set of projects underway (for example, Open311), an active and engaged set of city and other government collaborators (NYC, San Francisco, Washington, Seattle, Boston, Chicago, Los Angeles, just to name some big ones), a growing community of motivated, civic-minded technologists, and a set of plans to broaden participation, especially by smaller cities and towns and those outside the U.S.
We also have a fantastic set of launch partners, including our major launch funder, the Omidyar Network; our incubator organizations, Code for America and OpenPlans; and our stalwart supporter, O’Reilly Media. (Huge thanks for their extraordinary contributions.)
Whether you are a mayor, a city CIO, a budding technologist, an open source veteran, a civic activist, an entrepreneur, or just an interested citizen, we hope that you will get inspired and get involved in Civic Commons. Check out our projects, contribute to our wiki, join our mailing lists, orcontact us directly.
Together, we can tap everyday technological innovation to make government work better and cost less.
(For more information about Civic Commons, read the full release for the announcement.)
[This ran originally as my first blog post at civiccommons.org.]
Time-lapse videos of the night sky are mind-blowing: they convey the motion of the earth against the vast and distant firmament in which we live. Here's one of the most spectacular I've seen, filmed by two astronomers at the European Southern Observatory's Very Large Telescope in the Atacama Desert of Chile.
(OK, ignore the cheezy music, but marvel at how the observatory's laser guide, which creates an artificial star 90km above the surface of the earth that helps the telescopes correct for the blurring effect of our atmosphere, follows the movement of the stars).
And here is what's described as a "minor edit", in which the stars are fixed in place, and the rotation of the earth becomes evident. The effect is dramatic and astounding.
MIT's Media Lab has announced that Joichi Ito will be its new director. Many on the Internet are raising their glasses with rowdy cheers. In choosing Joi, the Media Lab has made a daring, inspired, energizing, and one might even say subversive move. It's particularly remarkable because Joi hasn't actually gotten around to graduating from college yet. Instead, he's been wasting his time as a DJ, ISP CEO, search engine CEO, angel investor and venture capitalist (think: Flickr, Six Apart, Last.fm, Kongregate, Kickstarter, and Twitter), CEO of Creative Commons, and board member at ICANN, Mozilla, WITNESS, Public Knowledge, and Global Voices.
Most importantly, Joi is a deeply ethical soul who devotes vast (and uncompensated) energy to improving the world he inhabits -- provoking and sustaining entrepreneurship and innovation, particularly outside Silicon Valley; building communities of common interest across national, cultural, linguistic, generational, and class frontiers; and tending loyally to his circle of friends around the world. (Or, in the case of Joi's World of Warcraft guild, doing all of that at once.) Joi combines raw brainpower with tireless energy, boundless curiosity, fearless introspection, and serendipitous creativity, all alloyed together atop his foundational commitment to values like openness, integrity, liberty, community, and the transformative, democratizing power of technology.
The celebratory outpouring over Joi's new appointment is a testament to his many good works, his great character, and to the Media Lab's laudable ambition, courage, and great good sense. It's so unusual -- and so bracing -- to see an institution of long standing, particularly an academic center, boldly embrace risk and invite creative disruption.
Congratulations, Joi & Media Lab!
Assessing Egypt's Echoes: How to Check for Yourself What's Happening with the Internet in Another Country
As anti-regime demonstrations have been taking place in Iran, Yemen, Bahrain, Jordan, and Algeria, are we seeing a tidal shockwave from Egypt’s people power revolution ripple across the Middle East and North Africa? And what lessons are the region's besieged regimes drawing from Egypt?
The latest reports from the ground:
- Iran: Massive street protests across Tehran and Isfahan on Monday, met with riot police using tear gas. At least one killed. Green Movement leaders, including Moussavi and Karroubi, arrested or detained.
- Bahrain: More than 10,000 Bahrainis took part in Monday’s Day of Rage, and at least that number have gathered in the central Pearl Square. One demonstrator was killed by police on Monday, and another on Tuesday. King Hamad is promising reforms, including easing censorship of media and the Internet.
- Yemen: Six days of continuous protests, with demonstrators now encountering swarms of police batons. After 32 years in office, President Saleh is making populist-sounding concessions (and budgetary payoffs to the Army) to cling to power.
- Algeria: A weekend of demonstrations in Algiers resulted in hundreds of arrests. The Bouteflika government announced a coming end to the 20-year-old state of emergency.
- Jordan: Weeks of peaceful protests led King Abdullah to replace his prime minister and cabinet, bring opposition figures into government, and even invite the local Muslim Brotherhood branch to join. Amid the tumult, Jordanians have started to air previously unthinkable public criticisms of the royal family.
Amid the stream of conventional news reports, there are many unknowns: How many people have been pre-emptively arrested? What is happening outside the capitals? What is actually happening in the places where communications are being blocked or suppressed?
One thing we do know is that, contrary to a breathless and unsourced (and nutty -- “Facebook accounts deleted”?) story in the UK’s Telegraph (British newspapers will evidently print anything), Algeria has not so far shut down its Internet or mobile connections or imposed new website blocks. How do we know? Well, to get a rough snapshot, we can look at real-time Internet traffic and routing data, as well as first-hand reports via Twitter and Facebook.
Specifically, we can use a set of public, online tools to assess the state of Internet and mobile phone connections in each of those five countries.
Here are three handy ways you can test reports about the Internet being blocked in X or Y country.
1. Google Traffic Data
Google maintains a very useful site that generates graphs of near-real-time, normalized measurements of network traffic to various Google services (search, YouTube, Blogger, etc.) on a country-by-country basis. This means you can select a country and a Google service and generate a graph that shows relative traffic patterns over time.
For example, here’s the graph for Google search traffic to Egypt from January 2, 2011 to now:
The canyon-like flatline reflects the period in which Egypt’s Internet and mobile networks were shut down: no network means no Google searches. (Notice, by the way, how each day has two separate peaks -- one in the morning, followed a drop-off around lunch, then the afternoon peak, followed by a steep plunge at night. Fascinating, huh?)
We can pull up the same charts for Algeria:
In none of those countries do we see evidence of a significant disruption or blocking of Google searches. By itself Google traffic data indicates only that a country’s networks are generally operational (and whether Google is specifically being blocked); it doesn’t afford any insight into whether other sites like Facebook or services like Twitter are being blocked in a targeted way, nor does it reveal whether the Internet is being intentionally slowed down or degraded -- a technique pioneered by Iran and currently used by Bahrain to inhibit citizen communications and, perhaps, to prevent larger video files from being uploaded and shared. It also offers no information as to whether and to what extent a country’s Internet and mobile networks are subject to surveillance. (Reminder: Activists, use Tor!)
Besides search, two other Google services are regularly subject to politically-motivated blocking: YouTube and Blogger. To get a sense of a country’s Internet situation, it’s a good idea to check the traffic data for them as well. Click the following links to see the traffic data for YouTube (Egypt, Algeria, Bahrain, Iran, Jordan, Yemen) and Blogger (Egypt, Algeria, Bahrain, Iran, Jordan, Yemen).
Those data show one interesting anomaly: YouTube and Blogger traffic from Iran surged in the early morning hours of February 11. To wit:
One explanation could be that a block on those sites was lifted, at least in part. That would be surprising, given that news reports indicate that Iran is tightening, rather than easing, its Internet restrictions amid the recent street protests. Another explanation could be that Google implemented a new, better technique for pinpointing the source of traffic. If we look at the lower bar of the graph, though, which shows the long-term traffic trends since 2009, we can see that Iran has been attempting to block YouTube since the disputed presidential election in mid-2009, and that traffic levels appear to have just jumped back to pre-blocking levels. I’ve asked Google folks for any insight and will report back if I learn anything.
Bottom line: The Google traffic data indicate that there is currently no general Internet shutdown in Algeria, Bahrain, Egypt, Iran, Jordan, or Yemen.
2. Twitter updates
A second useful technique is to check the latest Twitter updates from the country in question, using a hashtag and some search parameters. For example, you can go to Twitter’s website and enter a search like “#Bahrain Internet”. A scan of the most recent updates will give a more nuanced picture of what’s happening on the ground.
At the moment, Bahraini Twitterers are reporting that their Internet connections are getting slower and slower, that some specific locations like the Sheraton Hotel are fully cut off, and that some sites are blocked. And indeed, Batelco, Bahrain’s largest telecom provider, just two hours ago released a vague statement apologizing to its customers for “any service degradation they may be experiencing with their broadband Internet service” and asking for “our customers’ cooperation and understanding while we strive to restore full services, which we hope to do as soon as possible.” In other words, something is happening to disrupt Internet service in Bahrain. The slow-down could conceivably be due to a massive surge in Internet use, but, in view of Bahrain’s reasonably high-quality network infrastructure, is more likely the result governmental action, either intentionally slowing Internet traffic or taking network elements offline as they hastily install new surveillance and monitoring equipment. At this point, we can’t tell exactly what’s going on.
Scanning through the latest Twitter updates, here’s a quick summary of what local users are reporting about their Internet connections:
- Egypt: Internet up and running normally.
- Algeria: Internet up and running normally.
- Bahrain: Internet connections are slow and getting slower, with some locations entirely offline. Unclear whether this is due to an overwhelming spike in use, intentional governmental throttling, or as a side effect to surveillance-related tinkering with the network.
- Iran: Severe but uneven disruptions to Internet and mobile phone connectivity. Reports that most of Tehran is cut off, and that news sites and VPN and webmail services are blocked.
- Jordan: Internet up and running normally.
- Yemen: Some user reports that, after two days of blocking, the Internet is back up. (Note that this is not consistent with the Google/YouTube/Blogger search traffic data, which may indicate sporadic blocking, or, e.g., mobile-network-only blocking). Many tweets reporting site-specifc filtering. The number of tweets from users actually inside Yemen appears to be so small that we can't draw any conclusions with much confidence. Indeed, it could be that there is a government effort to block Twitter inside Yemen, or just very few Twitter users.
3. Expert analyst blogs
It is also a good idea to check for timely blog posts from noted experts in network connectivity. Two that I always check are the Renesys blog (where the terrific James Cowie and his team post), and the Arbor Networks blog (where the terrific Craig Labovitz and his team post). While the posts are sporadic, the content on those two blogs is always high-quality and reliable, built on solid data and deep networking expertise.
These, then, are three easy, quick-and-dirty techniques for assessing the state of Internet and mobile connectivity in a given country. In a future post, I’ll explain how to use more complex, less user-friendly online resources like the global BGP routing tables and ISP route announcements to do more in-depth, fine-grained analysis of Internet traffic patterns.
This morning, I did an interview on the public radio show Marketplace on the techniques, effects, and implications of the Egyptian government's decision to shut down the Internet and mobile phone networks. As the write-up of the segment says:
In the wake of massive protests in Egypt, other countries have been paying attention and sometimes covering things up. The government of China is suppressing mentions of the uprising. Iran is blocking access to news sites, perhaps fearing renewed protests in their country.
That's not all that surprising. They've been filtering the Internet for a long time. What Egypt did -- yanking the Internet almost entirely offline -- that's new.
On today's show, we look at the precedent set by Egypt's decision to block most Internet and cell phone access in that country. Jonathan Zittrain joins us. He's co-founder and co-director of Harvard's Berkman Center for Internet and Society and a frequent guest on our show. Jonathan says that while Iran, China, Saudi Arabia, and other countries have means to filter the Internet, Egypt's online system has always been pretty open so when they wanted to block it, they had to go all the way.
Andrew McLaughlin also joins us. He's a former director of global public policy at Google and former deputy chief technology officer for the Obama administration. He says while it was incredibly difficult to get online within Egypt's borders, neighboring countries had no issues at all, even though in many cases, their traffic goes through Egyptian territory.
Andrew tells us that Egypt has worked hard to position itself as a trustworthy hub for underwater cable that goes through the Red Sea, connecting to nearby countries. That service has not been disrupted.
As for what this means for other countries facing similar protests, we just don't know yet since Egypt's actions were unprecedented. But as protests begin to gather in other countries (and keep in mind that Egypt's protests were fueled by the overthrow in Tunisia), it's likely more governments will need to decide what to do about the Internet.
Egypt's internet cutoff has failed in its central aim, but there may yet be further harms
As recently as a week ago, Egypt's internet was extraordinary in the Arab world for its freedom. For more than a decade, the regime has adhered to a hands-off policy, leaving unblocked everything from rumours about President Hosni Mubarak's health to videos of police beatings. Unlike most of its regional neighbours and other authoritarian regimes, Egypt's government never built or required sophisticated technical infrastructures of censorship. (Of course, the country has hardly been a paradise of free expression: the state security forces have vigorously suppressed dissent through surveillance, arbitrary detentions and relentless intimidation of writers and editors.)
Partly as a result of its liberal policies, Egypt became a hub for internet and mobile network investment, home to a thriving and competitive communications sector that pioneered free dial-up services, achieved impressive rates of access and use, and offered speedy wireless and broadband networks at relatively low prices. Indeed, Egypt is today one of the major crossing points for the underwater fibre-optic cables that interconnect the regions of the globe.
But last Thursday, the Mubarak regime shattered a decade's worth of accomplishment by issuing the order to shut down the mobile networks and internet links. Since the internet age dawned in the early 90s, no widely connected country had disconnected itself entirely. The starkness and suddenness of Egypt's reversal – from unrestricted to unreachable – marks one of the many tragedies of the Mubarak regime's brutal and hamfisted response to last week's emergence of citizen protests.
The internet cutoff shows how the details of infrastructure matter. Despite having no large-scale or centralised censorship apparatus, Egypt was still able to shut down its communications in a matter of minutes. This was possible because Egypt permitted only three wireless carriers to operate, and required all internet service providers (ISPs) to funnel their traffic through a handful of international links. Confronted with mass demonstrations and fearful about a populace able to organise itself, the government had to order fewer than a dozen companies to shut down their networks and disconnect their routers from the global internet.
The blackout has proved increasingly ineffective. A handful of networks have remained connected, including one independent ISP, the country's academic and research network, and a few major banks, businesses and government institutions. Whether these reflect deliberate defiance, privileged connections, or tactical exceptions --one might imagine, for example, that members of Mubarak's family and inner circle would want to have Internet access to move money, buy tickets, or make hotel reservations abroad — is as yet unknown.
Moreover, innovative Egyptians are finding ways to overcome the block. They are relaying information by voice, exploiting small and unnoticed openings in the digital firewall, and dusting off old modems to tap foreign dial-up services.
For democracies, one lesson here is clear: diversity and complexity in our network architectures is a very good thing. Likewise, enforcement of public policies such as network neutrality – the principle that access providers should not be permitted to control what their customers can do online – are important to prevent networks from installing tools and capabilities that could be abused in moments of crisis. For dictatorships and authoritarian regimes, however, the lesson will be quite the opposite.
A central unknown at this moment is what the economic harm to the country will be. Without internet and voice networks, Egyptians are losing transactions and deals, their stocks and commodities cannot be traded, their goods are halted on frozen transportation networks, and their bank deposits are beyond reach.
Also unknown is how many Egyptians have been harmed in non-economic ways – as human beings. As things stand, a worried mother who has not heard from her son or daughter can't send an email or check Facebook for a status update. A witness to violence or abuse can't seek help, document responsibility, or warn others via Twitter or a blog.
Life-saving information is inaccessible. Healthy, civil debate about the future is squashed. And in the absence of trustworthy news, firsthand reports and real-time images, rumour and fear flourish. In all those ways, the total internet cutoff undermines the government's own interest in restoring calm and order.
What is clear, however, is that the communications cutoff has failed in its central objective, which was to stop the Egyptian people from organising and mobilising in opposition. As hundreds of thousands of Egyptians gather on the streets to demand change, we must all hope that Egypt's officials and CEOs will see the writing on the wall, weigh the costs to Egyptian businesses and citizens – to their economic interests, family needs and human rights – and defy their president's unprecedented and increasingly ineffective blackout.
Originally printed and posted in The Guardian.