I really, really love Instapaper;  I use it daily for its clean aesthetic, its uncluttered reading experience, its super-well-honed features. So it's fair to say I'm pretty excited about this post by Marco Arment. Wired takes note.

Matt Langer wrote an interesting post yesterday questioning Digg for the way our iOS apps present stories.  On Apple mobile devices, the Digg app by default frames and pre-caches the stories our editors highlight so that users can read them offline -- on the subway or an airplane, for example.

Matt’s making a really important point that we take seriously.

Curatorial sites and mobile apps like Digg need to drive traffic back to publishers, along with the ad impressions and clicks that many depend on for monetization.  Digg has two goals: (a) provide a terrific experience for our users, and (b) drive loads of traffic to the best publishers, big and small, so that they can keep producing more great creative work. We want The Digg Effect to be welcome, as users discover not only individual pieces of great writing, graphics, and video, but great sites, sources, publishers, and authors.

And indeed, the Digg website -- where the vast majority of our traffic happens -- does not frame or pre-cache stories, but sends readers directly to the source webpages, replete with the ad units and monetization tools the publishers have deployed.  In the mobile environment, however, we got a raft of input from users to the effect that they wanted Digg to provide a good offline experience.  In response to that, we implemented pre-caching and an embedded frame that presents only the target story.  As Matt rightly points out, that fails to immediately drive impressions and clicks back to the publisher. 

Here’s the conundrum for mobile reading apps:  (1) we want to support and improve publisher monetization as effectively as possible;  and (2) we also want to give Digg app users a great offline experience;  but (3) the vast majority of publishers do not provide portable ad formats that we can grab, attach, and render in ways that enable the publisher to capture all the impressions and clicks their ads can earn. Here's one promising approach that we’ve been talking with some publisher friends about: Publishers provide standardized portable ad units that we can automatically ingest and position alongside their stories;  Digg and other mobile reading apps embrace those portable ad units and deliver impression data, including offline impressions.

We would love publishers to provide standardized, portable ad formats that will effectively monetize their content in mobile readers.  That’d be awesome.  To see if that's viable, we’re going to do a round of conversations with publishers to figure out whether they’ve actually already got something workable (for example, the ad units that accompany mobile web versions of their content, or that some ad platforms have inserted directly into RSS outputs), or whether we should join together to build and implement something new that’d work not only for Digg, but all the other mobile reading apps out there.

It should be possible -- easy, even -- for publishers to know that their ad units are accompanying their content, and being rendered as they want, regardless of whether the reader is reading via a web browser, smartphone browser, web-based reader, online mobile app, offline mobile app, tablet reading app, Kindle, etc.  That may require some rudimentary standardization of portable ad unit formats that can painlessly be created by publishers and automatically interpreted and implemented by all the various types of reading experiences.

The other piece of feedback that we hear from publishers is that they want to know who their readers are. This is a tricky technical challenge that very few publishers have the resources for, but it’s a solvable problem that we’ve been thinking about for a while. Ted Roden and Mike Young — the guys behind the first version of News.me — proposed this to the New York Times and it looks like they’re starting to roll it out now, in the form of NYT Everywhere.

In the meantime, we’re going to take a close look at our iOS experience to see what adjustments might do a better job of sending as many impressions and clicks back to publishers’ own sites as possible, as well as doing right by individual authors.  That may require a change in defaults or more granular user options, or maybe even something more extreme.  We'll keep you posted. 

Best fringe benefit of the excursion, besides hanging out with my awesome sister Meg, was seeing Beirut play "Santa Fe" as the sun set over Portland.

To get a sense of betaworks, check out the amazing list of companies it has invested in -- for example, Twitter, Tumblr, Airbnb, Branch, Everlane, ideeli, GroupMe, Groupon, Kickstarter, Path, Tweetdeck. Its studio companies include Digg, Bit.ly, Chartbeat, SocialFlow, and Findings, with others under construction. (I'll have more to say about what I'm actually working on in the not-too-distant future.)

Huge thanks and a fond farewell to Tumblr, David Karp, and all my former colleagues there.  I'm really proud of what my teams -- international, outreach, communications, community, editorial, user support, marketing -- pulled off since I joined last year. Personal highlights: the amazing Storyboard blog, the Brazil launch, the human-friendly terms of service and policy docs, new policies on self-harm, SxSW, the fight against SOPA, and the vast global cohort of new Tumblr blogs and partners we brought onboard. I'm especially grateful to everyone who joined those teams on my watch. Tumblr's a terrific company, and an important platform for creativity, free speech, and community.

In which Jacob and I discuss Tumblr, Facebook, Twitter, copyright, and the challenge (and imperative) of making money for social networks, content creators, and publishers.

In which Jacob and I talk about Google's (and my) experience in China, why foreign tech companies will almost certainly fail there, and why China's government is slowly losing its grip over information.

Here's the first part of a recent conversation with Slate’s Jacob Weisberg. This bit is on Tumblr’s founder, the ethos and juju of the platform he created, and how Tumblr’s approach to identity translates into positivity, creativity, and anti-trollery.

Here's my keynote, "Fight for the Future: Libraries, Tech Policy, and the Fate of Human Knowledge."

Andrew McLaughlin @ Library Technology Conference 2012 from Library Technology Conference on Vimeo.

 The Prezi is here.

Here's a lunchtime talk I did at betaworks on my experience working in the White House, why it was awesome, why it was, um, frustrating, why it's hard to achieve large-scale change in the U.S. federal bureaucracy, and more.

Let's go to the tape.

Beyond Conventions: Technology - The Next Frontier from Ford Foundation on Vimeo.

It can be hard to recognize how fast and how deeply information technology is changing our day-to-day lives, and how profound the implications can be. Every year, thanks to Moore’s Law, the speeds of our laptops and devices go up even as the cost of computing goes down.  Internet bandwidth gets ever faster and cheaper, and wireless connectivity ever more ubiquitous. As a result, the cost of creating, organizing, analyzing, and distributing information has plunged dramatically over the past two decades. Today, an ordinary individual can communicate instantly with any connected person anywhere in the world; she can broadcast her ideas globally, readable by anyone with an interest.

Thanks to the ever-improving economics of computing, a company like Facebook can offer hundreds of millions of individuals the ability to tap a vast computing infrastructure optimized to publish their writings, deliver their messages, and store vast and growing oceans of multi-megapixel images — and all for free, funded by tiny incremental payments from advertisers. That’s how efficient and cheap computing power has become.

To put that in context, consider this: It currently costs about $150 to buy a hard disk with a terabyte of data storage, and it sits on the corner of your desk; as recently as the early 1990s, you would have had to spend over $1.5 million to buy the same thing, and it would have taken up a full corner of your room.

Or this:  The iPad 2 is roughly 1,500 times faster than the CDC 6600, NASA’s fastest supercomputer in 1969. The cost of an iPad 2 is currently $499; the cost of a CDC 6600 in 1969 was roughly $58,000,000, in inflation-adjusted 2010 dollars.

Amazing, right? But here’s the problem:  As William Gibson puts it, “the future is already here — it’s just not evenly distributed.” In particular, our governments — the agencies and departments and legislatures and courts that our democratic processes have ordained to serve the public — are finding it extraordinarily difficult to understand, much less embrace, the possibilities created and imperatives imposed by the technological advances we take for granted in our private lives.

Civic Commons, the new initiative I’m pleased to be joining today, is an effort to answer that problem. We believe that governments — especially the cities, towns, and counties that are on the hook to deliver public services every day — can now take advantage of the same technologies and techniques that have generated such enormous efficiencies and enabled such impressive new services by private enterprise. In a digitally interconnected world, cities don’t have to operate in isolation.  They don’t have to reinvent (or re-procure) the wheel every time they face a problem that technology could help address. Cities can pool their resources — their talents and ever-shrinking budgets — to build shared technologies. Just as open standards (e.g., the Internet protocols), shared infrastructures (e.g., cloud computing), and collaborative software (e.g., open source projects like Linux, Mozilla Firefox, and Apache Hadoop) have powered astonishing advances in personal and enterprise computing, it is now time for governments to put them to work for the public good.

We believe that governments can now build and deploy shared technologies — open standards, common infrastructures, collaborative projects, and open source software, together with proprietary systems — to improve public service delivery, transparency, accountability, public participation, and management effectiveness, all while spending less.

In sum, Civic Commons is built around two central convictions: first, that wave after wave of innovation is delivering amazing new capabilities to the people and organizations that can take advantage of them, and second, that, with a little help, governments can absolutely understand and seize the opportunities created by the rapid evolution of information technology.

So how are we going to do it? Civic Commons will operate as a neutral and expert non-profit that (a) helps cities and other governments understand the possibilities and pitfalls around shared technologies, (b) provides technical assistance, (c) facilitates the creation and management of collaborative technology projects, (d) connects interested cities with peers, collaborators, experts, vendors, and other supporters, and (e) creates high-quality information — such as guides, checklists, how-to’s, and a comprehensive catalog of civic technology — that is as comprehensible and useful to mayors, city managers, and citizens as it is to software engineers.

We’ve already got a great set of projects underway (for example, Open311), an active and engaged set of city and other government collaborators (NYC, San Francisco, Washington, Seattle, Boston, Chicago, Los Angeles, just to name some big ones), a growing community of motivated, civic-minded technologists, and a set of plans to broaden participation, especially by smaller cities and towns and those outside the U.S.

We also have a fantastic set of launch partners, including our major launch funder, the Omidyar Network;  our incubator organizations, Code for America and OpenPlans;  and our stalwart supporter, O’Reilly Media. (Huge thanks for their extraordinary contributions.)

Whether you are a mayor, a city CIO, a budding technologist, an open source veteran, a civic activist, an entrepreneur, or just an interested citizen, we hope that you will get inspired and get involved in Civic Commons. Check out our projects, contribute to our wiki, join our mailing lists, orcontact us directly.

Together, we can tap everyday technological innovation to make government work better and cost less.

Andrew McLaughlin
Executive Director
Civic Commons

(For more information about Civic Commons, read the full release for the announcement.)

[This ran originally as my first blog post at civiccommons.org.]

(OK, ignore the cheezy music, but marvel at how the observatory's laser guide, which creates an artificial star 90km above the surface of the earth that helps the telescopes correct for the blurring effect of our atmosphere, follows the movement of the stars).

Most importantly, Joi is a deeply ethical soul who devotes vast (and uncompensated) energy to improving the world he inhabits -- provoking and sustaining entrepreneurship and innovation, particularly outside Silicon Valley;  building communities of common interest across national, cultural, linguistic, generational, and class frontiers;  and tending loyally to his circle of friends around the world.  (Or, in the case of Joi's World of Warcraft guild, doing all of that at once.)  Joi combines raw brainpower with tireless energy, boundless curiosity, fearless introspection, and serendipitous creativity, all alloyed together atop his foundational commitment to values like openness, integrity, liberty, community, and the transformative, democratizing power of technology.

The celebratory outpouring over Joi's new appointment is a testament to his many good works, his great character, and to the Media Lab's laudable ambition, courage, and great good sense.  It's so unusual -- and so bracing -- to see an institution of long standing, particularly an academic center, boldly embrace risk and invite creative disruption.

Congratulations, Joi & Media Lab!

As anti-regime demonstrations have been taking place in Iran, Yemen, Bahrain, Jordan, and Algeria, are we seeing a tidal shockwave from Egypt’s people power revolution ripple across the Middle East and North Africa?  And what lessons are the region's besieged regimes drawing from Egypt?

The latest reports from the ground:

•  Iran:  Massive street protests across Tehran and Isfahan on Monday, met with riot police using tear gas.  At least one killed.  Green Movement leaders, including Moussavi and Karroubi, arrested or detained.
• Bahrain:  More than 10,000 Bahrainis took part in Monday’s Day of Rage, and at least that number have gathered in the central Pearl Square.  One demonstrator was killed by police on Monday, and another on Tuesday.  King Hamad is promising reforms, including easing censorship of media and the Internet.
• Yemen:  Six days of continuous protests, with demonstrators now encountering swarms of police batons.  After 32 years in office, President Saleh is making populist-sounding concessions (and budgetary payoffs to the Army) to cling to power.
• Algeria:  A weekend of demonstrations in Algiers resulted in hundreds of arrests.  The Bouteflika government announced a coming end to the 20-year-old state of emergency.
• Jordan: Weeks of peaceful protests led King Abdullah to replace his prime minister and cabinet, bring opposition figures into government, and even invite the local Muslim Brotherhood branch to join.  Amid the tumult, Jordanians have started to air previously unthinkable public criticisms of the royal family.

[Sources:  NYTimes, WPost. And the Atlantic is putting up occasional posts summarizing what we know about the situation in those counties.]

Amid the stream of conventional news reports, there are many unknowns:  How many people have been pre-emptively arrested?  What is happening outside the capitals?  What is actually happening in the places where communications are being blocked or suppressed?

One thing we do know is that, contrary to a breathless and unsourced (and nutty -- “Facebook accounts deleted”?) story in the UK’s Telegraph (British newspapers will evidently print anything), Algeria has not so far shut down its Internet or mobile connections or imposed new website blocks.  How do we know?  Well, to get a rough snapshot, we can look at real-time Internet traffic and routing data, as well as first-hand reports via Twitter and Facebook.  

Specifically, we can use a set of public, online tools to assess the state of Internet and mobile phone connections in each of those five countries.  

Here are three handy ways you can test reports about the Internet being blocked in X or Y country.

 1.  Google Traffic Data

Google maintains a very useful site that generates graphs of near-real-time, normalized measurements of network traffic to various Google services (search, YouTube, Blogger, etc.) on a country-by-country basis.  This means you can select a country and a Google service and generate a graph that shows relative traffic patterns over time. 

For example, here’s the graph for Google search traffic to Egypt from January 2, 2011 to now: 

The canyon-like flatline reflects the period in which Egypt’s Internet and mobile networks were shut down:  no network means no Google searches.  (Notice, by the way, how each day has two separate peaks -- one in the morning, followed a drop-off around lunch, then the afternoon peak, followed by a steep plunge at night.  Fascinating, huh?)

We can pull up the same charts for Algeria:

Bahrain: 

Iran: 

Jordan:

and Yemen: 

In none of those countries do we see evidence of a significant disruption or blocking of Google searches.  By itself Google traffic data indicates only that a country’s networks are generally operational (and whether Google is specifically being blocked);  it doesn’t afford any insight into whether other sites like Facebook or services like Twitter are being blocked in a targeted way, nor does it reveal whether the Internet is being intentionally slowed down or degraded -- a technique pioneered by Iran and currently used by Bahrain to inhibit citizen communications and, perhaps, to prevent larger video files from being uploaded and shared.  It also offers no information as to whether and to what extent a country’s Internet and mobile networks are subject to surveillance.  (Reminder: Activists, use Tor!)

Besides search, two other Google services are regularly subject to politically-motivated blocking:  YouTube and Blogger.  To get a sense of a country’s Internet situation, it’s a good idea to check the traffic data for them as well.  Click the following links to see the traffic data for YouTube (Egypt, Algeria, Bahrain, Iran, Jordan, Yemen) and Blogger (Egypt, Algeria, Bahrain, Iran, Jordan, Yemen).   

Those data show one interesting anomaly:  YouTube and Blogger traffic from Iran surged in the early morning hours of February 11.  To wit:

One explanation could be that a block on those sites was lifted, at least in part.  That would be surprising, given that news reports indicate that Iran is tightening, rather than easing, its Internet restrictions amid the recent street protests.  Another explanation could be that Google implemented a new, better technique for pinpointing the source of traffic.  If we look at the lower bar of the graph, though, which shows the long-term traffic trends since 2009, we can see that Iran has been attempting to block YouTube since the disputed presidential election in mid-2009, and that traffic levels appear to have just jumped back to pre-blocking levels. I’ve asked Google folks for any insight and will report back if I learn anything.

Bottom line:  The Google traffic data indicate that there is currently no general Internet shutdown in Algeria, Bahrain, Egypt, Iran, Jordan, or Yemen.

2.  Twitter updates 

A second useful technique is to check the latest Twitter updates from the country in question, using a hashtag and some search parameters.  For example, you can go to Twitter’s website and enter a search like “#Bahrain Internet”.  A scan of the most recent updates will give a more nuanced picture of what’s happening on the ground.

At the moment, Bahraini Twitterers are reporting that their Internet connections are getting slower and slower, that some specific locations like the Sheraton Hotel are fully cut off, and that some sites are blocked.  And indeed, Batelco, Bahrain’s largest telecom provider, just two hours ago released a vague statement apologizing to its customers for “any service degradation they may be experiencing with their broadband Internet service” and asking for “our customers’ cooperation and understanding while we strive to restore full services, which we hope to do as soon as possible.”  In other words, something is happening to disrupt Internet service in Bahrain.  The slow-down could conceivably be due to a massive surge in Internet use, but, in view of Bahrain’s reasonably high-quality network infrastructure, is more likely the result governmental action, either intentionally slowing Internet traffic or taking network elements offline as they hastily install new surveillance and monitoring equipment.  At this point, we can’t tell exactly what’s going on.

Scanning through the latest Twitter updates, here’s a quick summary of what local users are reporting about their Internet connections:

• Egypt:  Internet up and running normally.
• Algeria:  Internet up and running normally.
• Bahrain:  Internet connections are slow and getting slower, with some locations entirely offline.  Unclear whether this is due to an overwhelming spike in use, intentional governmental throttling, or as a side effect to surveillance-related tinkering with the network.
• Iran:  Severe but uneven disruptions to Internet and mobile phone connectivity.  Reports that most of Tehran is cut off, and that news sites and VPN and webmail services are blocked.
• Jordan:  Internet up and running normally.
• Yemen:  Some user reports that, after two days of blocking, the Internet is back up. (Note that this is not consistent with the Google/YouTube/Blogger search traffic data, which may indicate sporadic blocking, or, e.g., mobile-network-only blocking).  Many tweets reporting site-specifc filtering.  The number of tweets from users actually inside Yemen appears to be so small that we can't draw any conclusions with much confidence.  Indeed, it could be that there is a government effort to block Twitter inside Yemen, or just very few Twitter users.

3.  Expert analyst blogs

It is also a good idea to check for timely blog posts from noted experts in network connectivity.  Two that I always check are the Renesys blog (where the terrific James Cowie and his team post), and the Arbor Networks blog (where the terrific Craig Labovitz and his team post).  While the posts are sporadic, the content on those two blogs is always high-quality and reliable, built on solid data and deep networking expertise.

These, then, are three easy, quick-and-dirty techniques for assessing the state of Internet and mobile connectivity in a given country.  In a future post, I’ll explain how to use more complex, less user-friendly online resources like the global BGP routing tables and ISP route announcements to do more in-depth, fine-grained analysis of Internet traffic patterns. 

In the wake of massive protests in Egypt, other countries have been paying attention and sometimes covering things up. The government of China is suppressing mentions of the uprising. Iran is blocking access to news sites, perhaps fearing renewed protests in their country.

That's not all that surprising. They've been filtering the Internet for a long time. What Egypt did -- yanking the Internet almost entirely offline -- that's new.

On today's show, we look at the precedent set by Egypt's decision to block most Internet and cell phone access in that country. Jonathan Zittrain joins us. He's co-founder and co-director of Harvard's Berkman Center for Internet and Society and a frequent guest on our show. Jonathan says that while Iran, China, Saudi Arabia, and other countries have means to filter the Internet, Egypt's online system has always been pretty open so when they wanted to block it, they had to go all the way.

Andrew McLaughlin also joins us. He's a former director of global public policy at Google and former deputy chief technology officer for the Obama administration. He says while it was incredibly difficult to get online within Egypt's borders, neighboring countries had no issues at all, even though in many cases, their traffic goes through Egyptian territory.

Andrew tells us that Egypt has worked hard to position itself as a trustworthy hub for underwater cable that goes through the Red Sea, connecting to nearby countries. That service has not been disrupted.

As for what this means for other countries facing similar protests, we just don't know yet since Egypt's actions were unprecedented. But as protests begin to gather in other countries (and keep in mind that Egypt's protests were fueled by the overthrow in Tunisia), it's likely more governments will need to decide what to do about the Internet.

Here's a link to the full interview.  Here's the .mp3 of the show.