http://odeo.com/channels/3554-Jon-Udell no Strategies for Internet citizens Strategies for Internet citizens Strategies for Internet citizens en 40 Mon, 09 Nov 2009 07:08:41 -0800 Mon, 09 Nov 2009 07:08:41 -0800 http://odeo.com/episodes/25426119-Where-is-the-money-going Over the weekend I was poking around in the recipient-reported data at recovery.gov. I filtered the New Hampshire spreadsheet down to items for my town, Keene, and was a bit surprised to find no descriptions in many cases. Here’s the breakdown: # of awards 25 # of awards with descriptions 05 20% # of awards without descriptions 20 80% $ of awards 10,940,770 $ of awards with descriptions 1,260,719 12% $ of awards without descriptions 9,680,053 88% In this case, the half-dozen largest awards aren’t described: award amount funding agency recipient description EE00161 2,601,788 Sothwestern Community Services Inc S394A090030 1,471,540 Keene School District AIP #3-33-SBGP-06-2009 1,298,500 City of Keene 2W-33000209-0 1,129,608 City of Keene 2F-96102301-0 666,379 City of Keene 2F-96102301-0 655,395 City of Keene 0901NHCOS2 600,930 Sothwestern Community Services Inc 2009RKWX0608 459,850</td Department of Justice KEENE, CITY OF The COPS Hiring Recovery Program (CHRP) provides ... Over the weekend I was poking around in the recipient-reported data at recovery.gov. I filtered the New Hampshire spreadsheet down to items for my town, Keene, and was a bit surprised to find no descriptions in many cases. Here’s the breakdown: # of awards 25 # of awards with descriptions 05 20% # of awards without descriptions 20 80% $ of awards 10,940,770 $ of awards with descriptions 1,260,719 12% $ of awards without descriptions 9,680,053 88% In this case, the half-dozen largest awards aren’t described: award amount funding agency recipient description EE00161 2,601,788 Sothwestern Community Services Inc S394A090030 1,471,540 Keene School District AIP #3-33-SBGP-06-2009 1,298,500 City of Keene 2W-33000209-0 1,129,608 City of Keene 2F-96102301-0 666,379 City of Keene 2F-96102301-0 655,395 City of Keene 0901NHCOS2 600,930 Sothwestern Community Services Inc 2009RKWX0608 459,850</td Department of Justice KEENE, CITY OF The COPS Hiring Recovery Program (CHRP) provides funding directly to law enforcement agencies to hire and/or rehire career law enforcement officers in an effort to create and preserve jobs, and to increase their community policing capacity and crime prevention efforts. NH36S01050109 413,394</td Department of Housing and Urban Development KEENE HOUSING AUTHORITY ARRA Capital Fund Grant. Replacement of roofing, siding, and repair of exterior storage sheds on 29 public housing units at a family complex That got me wondering: Where does the money go? So I built a little app that explores ARRA awards for any city or town: http://elmcity.cloudapp.net/arra. For most places, it seems, the ratio of awards with descriptions to awards without isn’t quite so bad. In the case of Philadelphia, for example, “only” 27% of the dollars awarded ($280 million!) are not described. But even when the description field is filled in, how much does that tell us about what’s actually being done with the money? We can’t expect to find that information in a spreadsheet at recovery.gov. The knowledge is held collectively by the many people who are involved in the projects funded by these awards. If we want to materialize a view of that collective knowledge, the ARRA data provides a useful starting point. Every award is identified by an award number. These are, effectively, webscale identifiers — that is, more-or-less unique tags we could use to collate newspaper articles, blog entries, tweets, or any other online chatter about awards. To promote this idea, the app reports award numbers as search strings. In Keene, for example, the school district got an award for $1.47 million. The award number is S394A090030. If you search for that you’ll find nothing but a link back to a recovery.gov page entitled Where is the Money Going? Recovery.gov can’t bootstrap itself out of this circular trap. But if we use the tags that it has helpfully provided, we might be able to find out a lot more about where the money is going. Over the weekend I was poking around in the recipient-reported data at recovery.gov. I filtered the New Hampshire spreadsheet down to items for my town, Keene, and was a bit surprised to find no descriptions in many cases. Here’s the breakdown: # of awards 25 # of awards with descriptions 05 20% # of awards without descriptions 20 80% $ of awards 10,940,770 $ of awards with descriptions 1,260,719 12% $ of awards without descriptions 9,680,053 88% In this case, the half-dozen largest awards aren’t described: award amount funding agency recipient description EE00161 2,601,788 Sothwestern Community Services Inc S394A090030 1,471,540 Keene School District AIP #3-33-SBGP-06-2009 1,298,500 City of Keene 2W-33000209-0 1,129,608 City of Keene 2F-96102301-0 666,379 City of Keene 2F-96102301-0 655,395 City of Keene 0901NHCOS2 600,930 Sothwestern Community Services Inc 2009RKWX0608 459,850</td Department of Justice KEENE, CITY OF The COPS Hiring Recovery Program (CHRP) provides funding directly to law enforcement agencies to hire and/or rehire career law enforcement officers in an effort to create and preserve jobs, and to increase their community policing capacity and crime prevention efforts. NH36S01050109 413,394</td Department of Housing and Urban Development KEENE HOUSING AUTHORITY ARRA Capital Fund Grant. Replacement of roofing, siding, and repair of exterior storage sheds on 29 public housing units at a family complex That got me wondering: Where does the money go? So I built a little app that explores ARRA awards for any city or town: http://elmcity.cloudapp.net/arra. For most places, it seems, the ratio of awards with descriptions to awards without isn’t quite so bad. In the case of Philadelphia, for example, “only” 27% of the dollars awarded ($280 million!) are not described. But even when the description field is filled in, how much does that tell us about what’s actually being done with the money? We can’t expect to find that information in a spreadsheet at recovery.gov. The knowledge is held collectively by the many people who are involved in the projects funded by these awards. If we want to materialize a view of that collective knowledge, the ARRA data provides a useful starting point. Every award is identified by an award number. These are, effectively, webscale identifiers — that is, more-or-less unique tags we could use to collate newspaper articles, blog entries, tweets, or any other online chatter about awards. To promote this idea, the app reports award numbers as search strings. In Keene, for example, the school district got an award for $1.47 million. The award number is S394A090030. If you search for that you’ll find nothing but a link back to a recovery.gov page entitled Where is the Money Going? Recovery.gov can’t bootstrap itself out of this circular trap. But if we use the tags that it has helpfully provided, we might be able to find out a lot more about where the money is going. tag:odeo.com,2009-11-09,25426119 Mon, 09 Nov 2009 07:08:41 -0800 no Jon Udell Uncategorized http://odeo.com/episodes/25400239-Talking-with-Marco-Barulli-about-zero-knowledge-online-password-management A couple of years ago I was enamored with a clever password manager that pointed the way toward an ideal solution. It was really just a bookmarklet — a small chunk of JavaScript code — that used a simple method to produce a unique and strong password for the website you were visiting. The method was to combine a passphrase that you could remember with the domain name of the site, using a one-way cryptographic hash, in order to produce a strong password that would be unique to the site — and that you’d otherwise never be able to remember. It wasn’t perfect. Sometimes the passwords it generated wouldn’t meet a site’s requirements. And sometimes the login domain name would vary, which broke the scheme. But it introduced me to two powerful — and related — ideas. JavaScript could turn your browser into a programmable cryptographic engine. And that engine could be used to implement protocols that relied on cryptography but transmitted ... A couple of years ago I was enamored with a clever password manager that pointed the way toward an ideal solution. It was really just a bookmarklet — a small chunk of JavaScript code — that used a simple method to produce a unique and strong password for the website you were visiting. The method was to combine a passphrase that you could remember with the domain name of the site, using a one-way cryptographic hash, in order to produce a strong password that would be unique to the site — and that you’d otherwise never be able to remember. It wasn’t perfect. Sometimes the passwords it generated wouldn’t meet a site’s requirements. And sometimes the login domain name would vary, which broke the scheme. But it introduced me to two powerful — and related — ideas. JavaScript could turn your browser into a programmable cryptographic engine. And that engine could be used to implement protocols that relied on cryptography but transmitted no secrets over the wire. To my way of thinking, that’s a killer combination. For years I’ve been using Bruce Schneier’s Password Safe, a Windows program that keeps my passwords in an encrypted store. There are many such programs, another example being 1Password for the Mac. This kind of app lives on your computer and talks to a local data store. That means it’s cumbersome to move the app and your data from one of your machines to another. And you can’t use it online, say from a public machine at the library or a friend’s computer. Imagine a web application that would encrypt your credentials and store them in the cloud. It would deliver that encrypted store to any browser you happen to be using, along with a JavaScript engine that could decrypt it, display your credentials, and even use them to automatically log you onto any of your password-protected services. You’d trust it because its cryptographic code would be available for security pros to validate. I’ve wanted this solution for a long time. Now I have it: Clipperz. My guest for this week’s Innovators show is Marco Barulli, founder and CEO of Clipperz, which he describes as a zero-knowledge web application. What Clipperz has zero knowledge of is you and your data. It just connects you with your data, on terms that you control, in a way that reminds me of Peter Wayner’s concept of translucent databases. Clipperz is immediately useful to all of us who struggle to manage our growing collections of online credentials, But it’s also a great example of an important design principle. We reflexively build services that identity users and retain all kinds of information about them. Often we need such knowledge, but it’s a liability for the operators of services that store it, and a risk for users of those services. If it’s feasible not to know, we can embrace that constraint and achieve powerful effects. A couple of years ago I was enamored with a clever password manager that pointed the way toward an ideal solution. It was really just a bookmarklet — a small chunk of JavaScript code — that used a simple method to produce a unique and strong password for the website you were visiting. The method was to combine a passphrase that you could remember with the domain name of the site, using a one-way cryptographic hash, in order to produce a strong password that would be unique to the site — and that you’d otherwise never be able to remember. It wasn’t perfect. Sometimes the passwords it generated wouldn’t meet a site’s requirements. And sometimes the login domain name would vary, which broke the scheme. But it introduced me to two powerful — and related — ideas. JavaScript could turn your browser into a programmable cryptographic engine. And that engine could be used to implement protocols that relied on cryptography but transmitted no secrets over the wire. To my way of thinking, that’s a killer combination. For years I’ve been using Bruce Schneier’s Password Safe, a Windows program that keeps my passwords in an encrypted store. There are many such programs, another example being 1Password for the Mac. This kind of app lives on your computer and talks to a local data store. That means it’s cumbersome to move the app and your data from one of your machines to another. And you can’t use it online, say from a public machine at the library or a friend’s computer. Imagine a web application that would encrypt your credentials and store them in the cloud. It would deliver that encrypted store to any browser you happen to be using, along with a JavaScript engine that could decrypt it, display your credentials, and even use them to automatically log you onto any of your password-protected services. You’d trust it because its cryptographic code would be available for security pros to validate. I’ve wanted this solution for a long time. Now I have it: Clipperz. My guest for this week’s Innovators show is Marco Barulli, founder and CEO of Clipperz, which he describes as a zero-knowledge web application. What Clipperz has zero knowledge of is you and your data. It just connects you with your data, on terms that you control, in a way that reminds me of Peter Wayner’s concept of translucent databases. Clipperz is immediately useful to all of us who struggle to manage our growing collections of online credentials, But it’s also a great example of an important design principle. We reflexively build services that identity users and retain all kinds of information about them. Often we need such knowledge, but it’s a liability for the operators of services that store it, and a risk for users of those services. If it’s feasible not to know, we can embrace that constraint and achieve powerful effects. tag:odeo.com,2009-11-01,25400239 Sun, 01 Nov 2009 23:51:24 -0800 no Jon Udell Uncategorized http://odeo.com/episodes/25356414-A-literary-appreciation-of-the-Olson-Zoneinfo-tz-database You will probably never need to know about the Olson database, also known as the Zoneinfo or tz database. And were it not for my elmcity project I never would have looked into it. I knew roughly that this bedrock database is a compendium of definitions of the world’s timezones, plus rules for daylight savings transitions (DST), used by many operating systems and programming languages. I presumed that it was written Unix-style, in some kind of plain-text format, and that’s true. Here, for example, are top-level DST rules for the United States since 1918: # Rule NAME FROM TO IN ON AT SAVE LETTER/S Rule US 1918 1919 Mar lastSun 2:00 1:00 D Rule US 1918 1919 Oct lastSun 2:00 0 S Rule US 1942 only Feb 9 2:00 1:00 W # War Rule US 1945 only Aug 14 23:00u 1:00 P # Peace Rule US 1945 only Sep 30 2:00 0 S Rule US 1967 2006 Oct lastSun 2:00 0 S Rule US 1967 1973 Apr lastSun 2:00 1:00 D Rule US 1974 only Jan 6 2:00 1:00 D Rule US 1975 only Feb 23 2:00 1:00 D Rule US 1976 1986 Apr la... You will probably never need to know about the Olson database, also known as the Zoneinfo or tz database. And were it not for my elmcity project I never would have looked into it. I knew roughly that this bedrock database is a compendium of definitions of the world’s timezones, plus rules for daylight savings transitions (DST), used by many operating systems and programming languages. I presumed that it was written Unix-style, in some kind of plain-text format, and that’s true. Here, for example, are top-level DST rules for the United States since 1918: # Rule NAME FROM TO IN ON AT SAVE LETTER/S Rule US 1918 1919 Mar lastSun 2:00 1:00 D Rule US 1918 1919 Oct lastSun 2:00 0 S Rule US 1942 only Feb 9 2:00 1:00 W # War Rule US 1945 only Aug 14 23:00u 1:00 P # Peace Rule US 1945 only Sep 30 2:00 0 S Rule US 1967 2006 Oct lastSun 2:00 0 S Rule US 1967 1973 Apr lastSun 2:00 1:00 D Rule US 1974 only Jan 6 2:00 1:00 D Rule US 1975 only Feb 23 2:00 1:00 D Rule US 1976 1986 Apr lastSun 2:00 1:00 D Rule US 1987 2006 Apr Sun>=1 2:00 1:00 D Rule US 2007 max Mar Sun>=8 2:00 1:00 D Rule US 2007 max Nov Sun>=1 2:00 0 S What I didn’t appreciate, until I finally unzipped and untarred a copy of ftp://elsie.nci.nih.gov/pub/tzdata2009o.tar.gz, is the historical scholarship scribbled in the margins of this remarkable database, or document, or hybrid of the two. You can see a glimpse of that scholarship in the above example. The most recent two rules define the latest (2007) change to US daylight savings. The spring forward rule says: “On the second Sunday in March, at 2AM, save one hour, and use D to change EST to EDT.” Likewise, on the fast-approaching first Sunday in November, spend one hour and go back to EST. But look at the rules for Feb 9 1942 and Aug 14 1945. The letters are W and P instead of D and S. And the comments tell us that during that period there were timezones like Eastern War Time (EWT) and Eastern Peace Time (EPT). Arthur David Olson elaborates: From Arthur David Olson (2000-09-25): Last night I heard part of a rebroadcast of a 1945 Arch Oboler radio drama. In the introduction, Oboler spoke of “Eastern Peace Time.” An AltaVista search turned up :”When the time is announced over the radio now, it is ‘Eastern Peace Time’ instead of the old familiar ‘Eastern War Time.’ Peace is wonderful.”   Most of this Talmudic scholarship comes from founding contributor Arthur David Olson and editor Paul Eggert, both of whose Wikipedia pages, although referenced from the Zoneinfo page, strangely do not exist. But the Olson/Eggert commentary is also interspersed with many contributions, like this one about the Mount Washington Observatory. From Dave Cantor (2004-11-02) Early this summer I had the occasion to visit the Mount Washington Observatory weather station atop (of course!) Mount Washington [, NH]…. One of the staff members said that the station was on Eastern Standard Time and didn’t change their clocks for Daylight Saving … so that their reports will always have times which are 5 hours behind UTC.   Since Mount Washington has a climate all its own, I guess it makes sense for it to have its own time as well. Here’s a glimpse of Alaska’s timezone history: From Paul Eggert (2001-05-30): Howse writes that Alaska switched from the Julian to the Gregorian calendar, and from east-of-GMT to west-of-GMT days, when the US bought it from Russia. This was on 1867-10-18, a Friday; the previous day was 1867-10-06 Julian, also a Friday. Include only the time zone part of this transition, ignoring the switch from Julian to Gregorian, since we can’t represent the Julian calendar. As far as we know, none of the exact locations mentioned below were permanently inhabited in 1867 by anyone using either calendar. (Yakutat was colonized by the Russians in 1799, but the settlement was destroyed in 1805 by a Yakutat-kon war party.) However, there were nearby inhabitants in some cases and for our purposes perhaps it’s best to simply use the official transition.   You have to have a sense of humor about this stuff, and Paul Eggert does: From Paul Eggert (1999-03-31): Shanks writes that Michigan started using standard time on 1885-09-18, but Howse writes (pp 124-125, referring to Popular Astronomy, 1901-01) that Detroit kept local time until 1900 when the City Council decreed that clocks should be put back twenty-eight minutes to Central Standard Time. Half the city obeyed, half refused. After considerable debate, the decision was rescinded and the city reverted to Sun time. A derisive offer to erect a sundial in front of the city hall was referred to the Committee on Sewers. Then, in 1905, Central time was adopted by city vote.   This story is too entertaining to be false, so go with Howse over Shanks.   The document is chock full of these sorts of you-can’t-make-this-stuff-up tales: From Paul Eggert (2001-03-06), following a tip by Markus Kuhn: Pam Belluck reported in the New York Times (2001-01-31) that the Indiana Legislature is considering a bill to adopt DST statewide. Her article mentioned Vevay, whose post office observes a different time zone from Danner’s Hardware across the street.   I love this one about the cranky Portuguese prime minister: Martin Bruckmann (1996-02-29) reports via Peter Ilieve that Portugal is reverting to 0:00 by not moving its clocks this spring. The new Prime Minister was fed up with getting up in the dark in the winter.   Of course Gaza could hardly fail to exhibit weirdness: From Ephraim Silverberg (1997-03-04, 1998-03-16, 1998-12-28, 2000-01-17 and 2000-07-25): According to the Office of the Secretary General of the Ministry of Interior, there is NO set rule for Daylight-Savings/Standard time changes. One thing is entrenched in law, however: that there must be at least 150 days of daylight savings time annually.   The rule names for this zone are poignant too: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Asia/Gaza 2:17:52 - LMT 1900 Oct 2:00 Zion EET 1948 May 15 2:00 EgyptAsia EE%sT 1967 Jun 5 2:00 Zion I%sT 1996 2:00 Jordan EE%sT 1999 2:00 Palestine EE%sT There’s also some wonderful commentary in the various software libraries that embody the Olson database. Here’s Stuart Bishop on why pytz, the Python implementation, supports almost all of the Olson timezones: As Saudi Arabia gave up trying to cope with their timezone definition, I see no reason to complicate my code further to cope with them. (I understand the intention was to set sunset to 0:00 local time, the start of the Islamic day. In the best case caused the DST offset to change daily and worst case caused the DST offset to change each instant depending on how you interpreted the ruling.)   It’s all deliciously absurd. And according to Paul Eggert, Ben Franklin is having the last laugh: From Paul Eggert (2001-03-06): Daylight Saving Time was first suggested as a joke by Benjamin Franklin in his whimsical essay “An Economical Project for Diminishing the Cost of Light” published in the Journal de Paris (1784-04-26). Not everyone is happy with the results.   So is Olson/Zoneinfo/tz a database or a document? Clearly both. And its synthesis of the two modes is, I would argue, a nice example of literate programming. You will probably never need to know about the Olson database, also known as the Zoneinfo or tz database. And were it not for my elmcity project I never would have looked into it. I knew roughly that this bedrock database is a compendium of definitions of the world’s timezones, plus rules for daylight savings transitions (DST), used by many operating systems and programming languages. I presumed that it was written Unix-style, in some kind of plain-text format, and that’s true. Here, for example, are top-level DST rules for the United States since 1918: # Rule NAME FROM TO IN ON AT SAVE LETTER/S Rule US 1918 1919 Mar lastSun 2:00 1:00 D Rule US 1918 1919 Oct lastSun 2:00 0 S Rule US 1942 only Feb 9 2:00 1:00 W # War Rule US 1945 only Aug 14 23:00u 1:00 P # Peace Rule US 1945 only Sep 30 2:00 0 S Rule US 1967 2006 Oct lastSun 2:00 0 S Rule US 1967 1973 Apr lastSun 2:00 1:00 D Rule US 1974 only Jan 6 2:00 1:00 D Rule US 1975 only Feb 23 2:00 1:00 D Rule US 1976 1986 Apr lastSun 2:00 1:00 D Rule US 1987 2006 Apr Sun>=1 2:00 1:00 D Rule US 2007 max Mar Sun>=8 2:00 1:00 D Rule US 2007 max Nov Sun>=1 2:00 0 S What I didn’t appreciate, until I finally unzipped and untarred a copy of ftp://elsie.nci.nih.gov/pub/tzdata2009o.tar.gz, is the historical scholarship scribbled in the margins of this remarkable database, or document, or hybrid of the two. You can see a glimpse of that scholarship in the above example. The most recent two rules define the latest (2007) change to US daylight savings. The spring forward rule says: “On the second Sunday in March, at 2AM, save one hour, and use D to change EST to EDT.” Likewise, on the fast-approaching first Sunday in November, spend one hour and go back to EST. But look at the rules for Feb 9 1942 and Aug 14 1945. The letters are W and P instead of D and S. And the comments tell us that during that period there were timezones like Eastern War Time (EWT) and Eastern Peace Time (EPT). Arthur David Olson elaborates: From Arthur David Olson (2000-09-25): Last night I heard part of a rebroadcast of a 1945 Arch Oboler radio drama. In the introduction, Oboler spoke of “Eastern Peace Time.” An AltaVista search turned up :”When the time is announced over the radio now, it is ‘Eastern Peace Time’ instead of the old familiar ‘Eastern War Time.’ Peace is wonderful.”   Most of this Talmudic scholarship comes from founding contributor Arthur David Olson and editor Paul Eggert, both of whose Wikipedia pages, although referenced from the Zoneinfo page, strangely do not exist. But the Olson/Eggert commentary is also interspersed with many contributions, like this one about the Mount Washington Observatory. From Dave Cantor (2004-11-02) Early this summer I had the occasion to visit the Mount Washington Observatory weather station atop (of course!) Mount Washington [, NH]…. One of the staff members said that the station was on Eastern Standard Time and didn’t change their clocks for Daylight Saving … so that their reports will always have times which are 5 hours behind UTC.   Since Mount Washington has a climate all its own, I guess it makes sense for it to have its own time as well. Here’s a glimpse of Alaska’s timezone history: From Paul Eggert (2001-05-30): Howse writes that Alaska switched from the Julian to the Gregorian calendar, and from east-of-GMT to west-of-GMT days, when the US bought it from Russia. This was on 1867-10-18, a Friday; the previous day was 1867-10-06 Julian, also a Friday. Include only the time zone part of this transition, ignoring the switch from Julian to Gregorian, since we can’t represent the Julian calendar. As far as we know, none of the exact locations mentioned below were permanently inhabited in 1867 by anyone using either calendar. (Yakutat was colonized by the Russians in 1799, but the settlement was destroyed in 1805 by a Yakutat-kon war party.) However, there were nearby inhabitants in some cases and for our purposes perhaps it’s best to simply use the official transition.   You have to have a sense of humor about this stuff, and Paul Eggert does: From Paul Eggert (1999-03-31): Shanks writes that Michigan started using standard time on 1885-09-18, but Howse writes (pp 124-125, referring to Popular Astronomy, 1901-01) that Detroit kept local time until 1900 when the City Council decreed that clocks should be put back twenty-eight minutes to Central Standard Time. Half the city obeyed, half refused. After considerable debate, the decision was rescinded and the city reverted to Sun time. A derisive offer to erect a sundial in front of the city hall was referred to the Committee on Sewers. Then, in 1905, Central time was adopted by city vote.   This story is too entertaining to be false, so go with Howse over Shanks.   The document is chock full of these sorts of you-can’t-make-this-stuff-up tales: From Paul Eggert (2001-03-06), following a tip by Markus Kuhn: Pam Belluck reported in the New York Times (2001-01-31) that the Indiana Legislature is considering a bill to adopt DST statewide. Her article mentioned Vevay, whose post office observes a different time zone from Danner’s Hardware across the street.   I love this one about the cranky Portuguese prime minister: Martin Bruckmann (1996-02-29) reports via Peter Ilieve that Portugal is reverting to 0:00 by not moving its clocks this spring. The new Prime Minister was fed up with getting up in the dark in the winter.   Of course Gaza could hardly fail to exhibit weirdness: From Ephraim Silverberg (1997-03-04, 1998-03-16, 1998-12-28, 2000-01-17 and 2000-07-25): According to the Office of the Secretary General of the Ministry of Interior, there is NO set rule for Daylight-Savings/Standard time changes. One thing is entrenched in law, however: that there must be at least 150 days of daylight savings time annually.   The rule names for this zone are poignant too: # Zone NAME GMTOFF RULES FORMAT [UNTIL] Zone Asia/Gaza 2:17:52 - LMT 1900 Oct 2:00 Zion EET 1948 May 15 2:00 EgyptAsia EE%sT 1967 Jun 5 2:00 Zion I%sT 1996 2:00 Jordan EE%sT 1999 2:00 Palestine EE%sT There’s also some wonderful commentary in the various software libraries that embody the Olson database. Here’s Stuart Bishop on why pytz, the Python implementation, supports almost all of the Olson timezones: As Saudi Arabia gave up trying to cope with their timezone definition, I see no reason to complicate my code further to cope with them. (I understand the intention was to set sunset to 0:00 local time, the start of the Islamic day. In the best case caused the DST offset to change daily and worst case caused the DST offset to change each instant depending on how you interpreted the ruling.)   It’s all deliciously absurd. And according to Paul Eggert, Ben Franklin is having the last laugh: From Paul Eggert (2001-03-06): Daylight Saving Time was first suggested as a joke by Benjamin Franklin in his whimsical essay “An Economical Project for Diminishing the Cost of Light” published in the Journal de Paris (1784-04-26). Not everyone is happy with the results.   So is Olson/Zoneinfo/tz a database or a document? Clearly both. And its synthesis of the two modes is, I would argue, a nice example of literate programming. tag:odeo.com,2009-10-23,25356414 Fri, 23 Oct 2009 07:45:19 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25349378-More-Python-and-C-idioms-Finding-the-difference-between-two-list Recently I’ve posted two examples[1, 2] of Python idioms alongside corresponding C# idioms. It always intrigues me to look at the same concept through different lenses, and it seems to intrigue others as well, so here’s a third installment. Today’s example comes from a real scenario. I’ve recently added a feature to the elmcity service that enables curators to control their hubs by sending Twitter direct messages to the service. One method, GetDirectMessagesFromTwitter, calls the Twitter API and returns a list of direct messages sent to the elmcity service. Another method, GetDirectMessagesFromAzure, calls the Azure table storage API and returns a list of direct messages stored there. The difference between the two lists — if any — represents new messages to be processed. Here’s one take on Python and C# idioms for finding the difference between two lists: Python C# fetched_messages = TwitterApi.GetDirectMessagesFromTwitter(); stored_message... Recently I’ve posted two examples[1, 2] of Python idioms alongside corresponding C# idioms. It always intrigues me to look at the same concept through different lenses, and it seems to intrigue others as well, so here’s a third installment. Today’s example comes from a real scenario. I’ve recently added a feature to the elmcity service that enables curators to control their hubs by sending Twitter direct messages to the service. One method, GetDirectMessagesFromTwitter, calls the Twitter API and returns a list of direct messages sent to the elmcity service. Another method, GetDirectMessagesFromAzure, calls the Azure table storage API and returns a list of direct messages stored there. The difference between the two lists — if any — represents new messages to be processed. Here’s one take on Python and C# idioms for finding the difference between two lists: Python C# fetched_messages = TwitterApi.GetDirectMessagesFromTwitter(); stored_messages = GetDirectMessagesFromAzure(); diff = set(fetched_messages) - set(stored_messages) return list(diff) var fetched_messages = GetDirectMessagesFromTwitter(); var stored_messages = GetDirectMessagesFromAzure(); var diff = fetched_messages.Except(stored_messages); return diff.ToList(); I can’t decide which one I prefer. Python’s set arithmetic is mathematically pure. But C#’s noun-verb syntax is appealing too. Which do you prefer? And why? PS: The Python example above is slightly concocted. It won’t work as shown here because I’m modeling Twitter direct messages as .NET objects. IronPython can use those objects, but the set subtraction fails because the objects returned from the two API calls aren’t directly comparable. A real working example would add something like this: fetched_message_sigs = [x.text+x.datetime for x in fetched_messages] stored_message_sigs = [x.text+x.datetime for x in stored_messages] diff = list(set(fetched_message_sigs) - set(stored_message_sigs)) But that’s a detail that would only obscure the side-by-side I’m making here. Recently I’ve posted two examples[1, 2] of Python idioms alongside corresponding C# idioms. It always intrigues me to look at the same concept through different lenses, and it seems to intrigue others as well, so here’s a third installment. Today’s example comes from a real scenario. I’ve recently added a feature to the elmcity service that enables curators to control their hubs by sending Twitter direct messages to the service. One method, GetDirectMessagesFromTwitter, calls the Twitter API and returns a list of direct messages sent to the elmcity service. Another method, GetDirectMessagesFromAzure, calls the Azure table storage API and returns a list of direct messages stored there. The difference between the two lists — if any — represents new messages to be processed. Here’s one take on Python and C# idioms for finding the difference between two lists: Python C# fetched_messages = TwitterApi.GetDirectMessagesFromTwitter(); stored_messages = GetDirectMessagesFromAzure(); diff = set(fetched_messages) - set(stored_messages) return list(diff) var fetched_messages = GetDirectMessagesFromTwitter(); var stored_messages = GetDirectMessagesFromAzure(); var diff = fetched_messages.Except(stored_messages); return diff.ToList(); I can’t decide which one I prefer. Python’s set arithmetic is mathematically pure. But C#’s noun-verb syntax is appealing too. Which do you prefer? And why? PS: The Python example above is slightly concocted. It won’t work as shown here because I’m modeling Twitter direct messages as .NET objects. IronPython can use those objects, but the set subtraction fails because the objects returned from the two API calls aren’t directly comparable. A real working example would add something like this: fetched_message_sigs = [x.text+x.datetime for x in fetched_messages] stored_message_sigs = [x.text+x.datetime for x in stored_messages] diff = list(set(fetched_message_sigs) - set(stored_message_sigs)) But that’s a detail that would only obscure the side-by-side I’m making here. tag:odeo.com,2009-10-22,25349378 Thu, 22 Oct 2009 16:20:53 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25381050-More-Python-and-C-idioms-Finding-the-difference-between-two-lists Recently I’ve posted two examples[1, 2] of Python idioms alongside corresponding C# idioms. It always intrigues me to look at the same concept through different lenses, and it seems to intrigue others as well, so here’s a third installment. Today’s example comes from a real scenario. I’ve recently added a feature to the elmcity service that enables curators to control their hubs by sending Twitter direct messages to the service. One method, GetDirectMessagesFromTwitter, calls the Twitter API and returns a list of direct messages sent to the elmcity service. Another method, GetDirectMessagesFromAzure, calls the Azure table storage API and returns a list of direct messages stored there. The difference between the two lists — if any — represents new messages to be processed. Here’s one take on Python and C# idioms for finding the difference between two lists: Python C# fetched_messages = GetDirectMessagesFromTwitter(); stored_messages = GetDire... Recently I’ve posted two examples[1, 2] of Python idioms alongside corresponding C# idioms. It always intrigues me to look at the same concept through different lenses, and it seems to intrigue others as well, so here’s a third installment. Today’s example comes from a real scenario. I’ve recently added a feature to the elmcity service that enables curators to control their hubs by sending Twitter direct messages to the service. One method, GetDirectMessagesFromTwitter, calls the Twitter API and returns a list of direct messages sent to the elmcity service. Another method, GetDirectMessagesFromAzure, calls the Azure table storage API and returns a list of direct messages stored there. The difference between the two lists — if any — represents new messages to be processed. Here’s one take on Python and C# idioms for finding the difference between two lists: Python C# fetched_messages = GetDirectMessagesFromTwitter(); stored_messages = GetDirectMessagesFromAzure(); diff = set(fetched_messages) - set(stored_messages) return list(diff) var fetched_messages = GetDirectMessagesFromTwitter(); var stored_messages = GetDirectMessagesFromAzure(); var diff = fetched_messages.Except( stored_messages); return diff.ToList(); I can’t decide which one I prefer. Python’s set arithmetic is mathematically pure. But C#’s noun-verb syntax is appealing too. Which do you prefer? And why? PS: The Python example above is slightly concocted. It won’t work as shown here because I’m modeling Twitter direct messages as .NET objects. IronPython can use those objects, but the set subtraction fails because the objects returned from the two API calls aren’t directly comparable. A real working example would add something like this: fetched_message_sigs = [x.text+x.datetime for x in fetched_messages] stored_message_sigs = [x.text+x.datetime for x in stored_messages] diff = list(set(fetched_message_sigs) - set(stored_message_sigs)) But that’s a detail that would only obscure the side-by-side comparison I’m making here. Recently I’ve posted two examples[1, 2] of Python idioms alongside corresponding C# idioms. It always intrigues me to look at the same concept through different lenses, and it seems to intrigue others as well, so here’s a third installment. Today’s example comes from a real scenario. I’ve recently added a feature to the elmcity service that enables curators to control their hubs by sending Twitter direct messages to the service. One method, GetDirectMessagesFromTwitter, calls the Twitter API and returns a list of direct messages sent to the elmcity service. Another method, GetDirectMessagesFromAzure, calls the Azure table storage API and returns a list of direct messages stored there. The difference between the two lists — if any — represents new messages to be processed. Here’s one take on Python and C# idioms for finding the difference between two lists: Python C# fetched_messages = GetDirectMessagesFromTwitter(); stored_messages = GetDirectMessagesFromAzure(); diff = set(fetched_messages) - set(stored_messages) return list(diff) var fetched_messages = GetDirectMessagesFromTwitter(); var stored_messages = GetDirectMessagesFromAzure(); var diff = fetched_messages.Except( stored_messages); return diff.ToList(); I can’t decide which one I prefer. Python’s set arithmetic is mathematically pure. But C#’s noun-verb syntax is appealing too. Which do you prefer? And why? PS: The Python example above is slightly concocted. It won’t work as shown here because I’m modeling Twitter direct messages as .NET objects. IronPython can use those objects, but the set subtraction fails because the objects returned from the two API calls aren’t directly comparable. A real working example would add something like this: fetched_message_sigs = [x.text+x.datetime for x in fetched_messages] stored_message_sigs = [x.text+x.datetime for x in stored_messages] diff = list(set(fetched_message_sigs) - set(stored_message_sigs)) But that’s a detail that would only obscure the side-by-side comparison I’m making here. tag:odeo.com,2009-10-22,25381050 Thu, 22 Oct 2009 16:20:53 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25325383-To-elmcity-From-curator-Message-start Because I am lazy, curious, and evangelical, the elmcity service works in an unusual way. Anything that I can delegate to other services I do. So when curators add feeds to hubs, or modify the behavior of hubs, they do it by bookmarking and tagging URLs at delicious.com. It would be foolish to only keep that registry and configuration data in delicious, so I don’t, I persist it to Azure tables. But for now, I’m delegating the data entry interface to delicious. It’s a lazy approach, in the good sense of lazy. I don’t want to build my own data entry system unless I can add important value, and in this case I can’t. I’m also curious to see how far this approach can take us. As the project has evolved, so has the tag vocabulary spoken between curators and the service. It’s an easy and natural process, and I don’t see any roadblocks ahead. Finally, I’m evangelizing this way of doing things because I continue to think that more people ... Because I am lazy, curious, and evangelical, the elmcity service works in an unusual way. Anything that I can delegate to other services I do. So when curators add feeds to hubs, or modify the behavior of hubs, they do it by bookmarking and tagging URLs at delicious.com. It would be foolish to only keep that registry and configuration data in delicious, so I don’t, I persist it to Azure tables. But for now, I’m delegating the data entry interface to delicious. It’s a lazy approach, in the good sense of lazy. I don’t want to build my own data entry system unless I can add important value, and in this case I can’t. I’m also curious to see how far this approach can take us. As the project has evolved, so has the tag vocabulary spoken between curators and the service. It’s an easy and natural process, and I don’t see any roadblocks ahead. Finally, I’m evangelizing this way of doing things because I continue to think that more people should appreciate it. In this scenario I’ve delegated something else to delicious: authentication. My service doesn’t have its own user accounts. Instead, as the administrator of the service, I tell it to trust a specific set of delicious accounts. When one of those accounts bookmarks an iCalendar URL, and tags it in a particular way, the service regards that as an authenticated request to add the feed to that hub’s registry. Other requests that curators can make include: Make the radius for my hub 5 miles. Make my timezone Arizona. Make my CSS file to this URL. But here’s one that curators have wanted to make and couldn’t: I just added a feed or changed a configuration option. Please reprocess my hub ASAP. We could represent this message with a tag. Or we could use the rudimentary messaging system in delicious. But these approaches seemed awkward, and I rejected them. Well, why not Twitter? True, it means that curators who want to send messages to the service will now need accounts in two places. But if they don’t already have accounts on both delicious and Twitter, they can create them. And those accounts will serve them in a variety of ways, unlike a single-purpose account on elmcity. So, it’s done. As the curator for Keene, I’ve added the tag twitter=judell to the delicious account that controls the Keene hub. As the elmcity service periodically scans its designated set of delicious accounts, it follows any Twitter handle it isn’t already following. Those Twitter accounts can then send direct messages to the Twitter account of the elmcity service. For now there’s only one thing a curator can say to the service in a direct message — “start” — which means “please reprocess my hub ASAP.” But I’m sure the control vocabulary will evolve. And of course the service can use the channel to send notifications back to curators. Twitter is famously unreliable, but that should be OK for my purposes. We’re not controlling the space shuttle. If a message doesn’t get through to the service on the first or second try, it’ll get through eventually, and that’ll be good enough. Someday I may have to build a data entry system and an accounts system. Then again, maybe not. Meanwhile I’m going to keep exploring this lightweight approach. It’s effective and, not coincidentally, it’s fun. Because I am lazy, curious, and evangelical, the elmcity service works in an unusual way. Anything that I can delegate to other services I do. So when curators add feeds to hubs, or modify the behavior of hubs, they do it by bookmarking and tagging URLs at delicious.com. It would be foolish to only keep that registry and configuration data in delicious, so I don’t, I persist it to Azure tables. But for now, I’m delegating the data entry interface to delicious. It’s a lazy approach, in the good sense of lazy. I don’t want to build my own data entry system unless I can add important value, and in this case I can’t. I’m also curious to see how far this approach can take us. As the project has evolved, so has the tag vocabulary spoken between curators and the service. It’s an easy and natural process, and I don’t see any roadblocks ahead. Finally, I’m evangelizing this way of doing things because I continue to think that more people should appreciate it. In this scenario I’ve delegated something else to delicious: authentication. My service doesn’t have its own user accounts. Instead, as the administrator of the service, I tell it to trust a specific set of delicious accounts. When one of those accounts bookmarks an iCalendar URL, and tags it in a particular way, the service regards that as an authenticated request to add the feed to that hub’s registry. Other requests that curators can make include: Make the radius for my hub 5 miles. Make my timezone Arizona. Make my CSS file to this URL. But here’s one that curators have wanted to make and couldn’t: I just added a feed or changed a configuration option. Please reprocess my hub ASAP. We could represent this message with a tag. Or we could use the rudimentary messaging system in delicious. But these approaches seemed awkward, and I rejected them. Well, why not Twitter? True, it means that curators who want to send messages to the service will now need accounts in two places. But if they don’t already have accounts on both delicious and Twitter, they can create them. And those accounts will serve them in a variety of ways, unlike a single-purpose account on elmcity. So, it’s done. As the curator for Keene, I’ve added the tag twitter=judell to the delicious account that controls the Keene hub. As the elmcity service periodically scans its designated set of delicious accounts, it follows any Twitter handle it isn’t already following. Those Twitter accounts can then send direct messages to the Twitter account of the elmcity service. For now there’s only one thing a curator can say to the service in a direct message — “start” — which means “please reprocess my hub ASAP.” But I’m sure the control vocabulary will evolve. And of course the service can use the channel to send notifications back to curators. Twitter is famously unreliable, but that should be OK for my purposes. We’re not controlling the space shuttle. If a message doesn’t get through to the service on the first or second try, it’ll get through eventually, and that’ll be good enough. Someday I may have to build a data entry system and an accounts system. Then again, maybe not. Meanwhile I’m going to keep exploring this lightweight approach. It’s effective and, not coincidentally, it’s fun. tag:odeo.com,2009-10-21,25325383 Wed, 21 Oct 2009 10:41:02 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25317345-Restructuring-expert-attention-to-revive-the-lost-art-of-personal-customer-service Instead of mourning the lost art of personal customer service, I would rather celebrate examples that show it’s still possible. Yesterday I found two gems. First, Southwest Airlines. I had booked a round-trip flight and then needed to change to one-way. You can’t do that online. So I clenched my jaw, called customer service, and prepared for the long wait. Instead, this: IVR: “Would you like us to call you back in about 20 minutes?” Me: “Why…yes! Beep, beep, beep, beep, beep, beep, beep, #.” My jaw relaxed. Twenty or so minutes later, an agent called back and we made the change. Now the unclenched jaw morphed into a smile. Second, FindTape.com. I’m making interior storm windows and I need double-stick tape for the project. Which, sure, you can buy online. But the smorgasbord of choices is paralyzing. I wasted a half-hour trying to figure out which product would best suit my unusual application and made no progress whatsoever. Then, at ... Instead of mourning the lost art of personal customer service, I would rather celebrate examples that show it’s still possible. Yesterday I found two gems. First, Southwest Airlines. I had booked a round-trip flight and then needed to change to one-way. You can’t do that online. So I clenched my jaw, called customer service, and prepared for the long wait. Instead, this: IVR: “Would you like us to call you back in about 20 minutes?” Me: “Why…yes! Beep, beep, beep, beep, beep, beep, beep, #.” My jaw relaxed. Twenty or so minutes later, an agent called back and we made the change. Now the unclenched jaw morphed into a smile. Second, FindTape.com. I’m making interior storm windows and I need double-stick tape for the project. Which, sure, you can buy online. But the smorgasbord of choices is paralyzing. I wasted a half-hour trying to figure out which product would best suit my unusual application and made no progress whatsoever. Then, at FindTape.com, I read this: If you have a specific question related to which tape would work best in your application please fill out and submit the following fields so that we can have an appropriate representative get back in contact with you. A fellow named Kevin wrote back, we’ve have been discussing my options, and now I’m ready to buy. Both examples remind me of Michael Nielsen’s luminous phrase: the restructuring of expert attention. He coined it to define a new era of scientific collaboration, but it applies more broadly. We’ve been told that companies can’t afford to focus expert attention on customers. The truth, of course, is that they can’t afford not to. For a generation and more we’ve driven a wedge between people who have expertise with products and services and people who need that expertise. How’s that working for you? Me neither. It’s true that expert attention is a scarce resource. But we’re living through a Cambrian explosion of awareness networks and communication modes. Used adroitly, they can optimize the allocation of that scarce resource. Which is a fancy way of saying: Maybe personal customer service isn’t a lost art after all. Instead of mourning the lost art of personal customer service, I would rather celebrate examples that show it’s still possible. Yesterday I found two gems. First, Southwest Airlines. I had booked a round-trip flight and then needed to change to one-way. You can’t do that online. So I clenched my jaw, called customer service, and prepared for the long wait. Instead, this: IVR: “Would you like us to call you back in about 20 minutes?” Me: “Why…yes! Beep, beep, beep, beep, beep, beep, beep, #.” My jaw relaxed. Twenty or so minutes later, an agent called back and we made the change. Now the unclenched jaw morphed into a smile. Second, FindTape.com. I’m making interior storm windows and I need double-stick tape for the project. Which, sure, you can buy online. But the smorgasbord of choices is paralyzing. I wasted a half-hour trying to figure out which product would best suit my unusual application and made no progress whatsoever. Then, at FindTape.com, I read this: If you have a specific question related to which tape would work best in your application please fill out and submit the following fields so that we can have an appropriate representative get back in contact with you. A fellow named Kevin wrote back, we’ve have been discussing my options, and now I’m ready to buy. Both examples remind me of Michael Nielsen’s luminous phrase: the restructuring of expert attention. He coined it to define a new era of scientific collaboration, but it applies more broadly. We’ve been told that companies can’t afford to focus expert attention on customers. The truth, of course, is that they can’t afford not to. For a generation and more we’ve driven a wedge between people who have expertise with products and services and people who need that expertise. How’s that working for you? Me neither. It’s true that expert attention is a scarce resource. But we’re living through a Cambrian explosion of awareness networks and communication modes. Used adroitly, they can optimize the allocation of that scarce resource. Which is a fancy way of saying: Maybe personal customer service isn’t a lost art after all. tag:odeo.com,2009-10-20,25317345 Tue, 20 Oct 2009 06:27:51 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25290754-Allman-Brothers-Oct-14-Huntington-or-Nashville-A-parable-about-syndication-and-provenance Yesterday Bill Rawlinson, the elmcity curator for Huntington, WV, noticed something odd about an event that showed up on Eventful.com: Here’s the example: http://eventful.com/huntington/events/allman-brothers-/E0-001-020736056-0. It appears the Allman Brothers were in concert today, but I’m pretty sure they weren’t. I’m pretty sure they weren’t either. At AllmanBrothersBand.com it says they were in Nashville on October 14. But if that’s true, Eventful isn’t the only site that got it wrong date. So, apparently, did a number of event-gathering and ticket-selling sites. Here are couple of examples I found. In cases like these it’s hard to nail down the provenance of a “fact” such as Allman Brothers, Huntington WV, October 14 2009. There is clearly syndication going on, but who’s upstream and who’s downstream? How is the network of feeds interconnected? Which is the authoritative source? I know what the answer to al... Yesterday Bill Rawlinson, the elmcity curator for Huntington, WV, noticed something odd about an event that showed up on Eventful.com: Here’s the example: http://eventful.com/huntington/events/allman-brothers-/E0-001-020736056-0. It appears the Allman Brothers were in concert today, but I’m pretty sure they weren’t. I’m pretty sure they weren’t either. At AllmanBrothersBand.com it says they were in Nashville on October 14. But if that’s true, Eventful isn’t the only site that got it wrong date. So, apparently, did a number of event-gathering and ticket-selling sites. Here are couple of examples I found. In cases like these it’s hard to nail down the provenance of a “fact” such as Allman Brothers, Huntington WV, October 14 2009. There is clearly syndication going on, but who’s upstream and who’s downstream? How is the network of feeds interconnected? Which is the authoritative source? I know what the answer to all these questions should be. The Allman Brothers themselves should be the authoritative source, and everyone else should syndicate from them. If AllmanBrothersBand.com published its schedule as calendar data rather than as calendarish web pages, the organization could control the data. Was there originally a concert planned for Huntington on the 14th? I don’t know, but say for the sake of argument there was. The Allman Brothers calendarish web page cannot effectively propagate a change of plan. An iCalendar feed, on the other hand, could. But calendarish web page are almost never alternately available as machine-readable iCalendar data that can reliably syndicate. Looking under the covers, I see that AllmanBrothersBand.com is a PostNuke site. Are there calendar modules for PostNuke that export iCalendar? None of the ones that I found seem to. Why don’t more content management systems make event information available as useful data? Why do they instead advertise things like XHTML compliance and not-very-useful RSS feeds? Because, chicken-and-egg, nobody ever seems to expect an iCalendar feed. If we can change that expectation, a nice chunk of the real-world semantic web will fall into place. And it won’t require RDFa or SPARQL or ontologies. Just good old RFC2445, right under our noses the whole time, if only we would open our eyes and look. Yesterday Bill Rawlinson, the elmcity curator for Huntington, WV, noticed something odd about an event that showed up on Eventful.com: Here’s the example: http://eventful.com/huntington/events/allman-brothers-/E0-001-020736056-0. It appears the Allman Brothers were in concert today, but I’m pretty sure they weren’t. I’m pretty sure they weren’t either. At AllmanBrothersBand.com it says they were in Nashville on October 14. But if that’s true, Eventful isn’t the only site that got it wrong date. So, apparently, did a number of event-gathering and ticket-selling sites. Here are couple of examples I found. In cases like these it’s hard to nail down the provenance of a “fact” such as Allman Brothers, Huntington WV, October 14 2009. There is clearly syndication going on, but who’s upstream and who’s downstream? How is the network of feeds interconnected? Which is the authoritative source? I know what the answer to all these questions should be. The Allman Brothers themselves should be the authoritative source, and everyone else should syndicate from them. If AllmanBrothersBand.com published its schedule as calendar data rather than as calendarish web pages, the organization could control the data. Was there originally a concert planned for Huntington on the 14th? I don’t know, but say for the sake of argument there was. The Allman Brothers calendarish web page cannot effectively propagate a change of plan. An iCalendar feed, on the other hand, could. But calendarish web page are almost never alternately available as machine-readable iCalendar data that can reliably syndicate. Looking under the covers, I see that AllmanBrothersBand.com is a PostNuke site. Are there calendar modules for PostNuke that export iCalendar? None of the ones that I found seem to. Why don’t more content management systems make event information available as useful data? Why do they instead advertise things like XHTML compliance and not-very-useful RSS feeds? Because, chicken-and-egg, nobody ever seems to expect an iCalendar feed. If we can change that expectation, a nice chunk of the real-world semantic web will fall into place. And it won’t require RDFa or SPARQL or ontologies. Just good old RFC2445, right under our noses the whole time, if only we would open our eyes and look. tag:odeo.com,2009-10-15,25290754 Thu, 15 Oct 2009 08:18:54 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25279707-Talking-with-Daniel-Debow-about-using-Rypple-to-open-the-Johari-Window On this week’s Innovators show, with Daniel Debow of Rypple, I learned about a cognitive psychological tool called the Johari Window. Rypple focuses on the quadrant of the Johari Window at the intersection of “known to others” and “not known to self” — the so-called blind area. The company is dedicated to the proposition that if we can become more aware of what others know about us that we don’t, we can improve ourselves along various axes: personal, social, and — critically for Rypple’s business model — professionally. How do you gain that awareness? By asking questions like: Am I giving sufficiently clear guidance? or Do I interrupt people too often? You direct these questions to a set of people whose feedback you value. Rypple anonymizes their responses and, to the extent you buy into the service, provides a progressively capable framework within which to continue the dialogue. This is a great idea, and one of the very f... On this week’s Innovators show, with Daniel Debow of Rypple, I learned about a cognitive psychological tool called the Johari Window. Rypple focuses on the quadrant of the Johari Window at the intersection of “known to others” and “not known to self” — the so-called blind area. The company is dedicated to the proposition that if we can become more aware of what others know about us that we don’t, we can improve ourselves along various axes: personal, social, and — critically for Rypple’s business model — professionally. How do you gain that awareness? By asking questions like: Am I giving sufficiently clear guidance? or Do I interrupt people too often? You direct these questions to a set of people whose feedback you value. Rypple anonymizes their responses and, to the extent you buy into the service, provides a progressively capable framework within which to continue the dialogue. This is a great idea, and one of the very few appropriate uses for online anonymity that I can imagine. Rypple, as a company, lives at the intersection of a couple of key trends. Social media, obviously, but also the services ecosystem. As we discuss in the podcast, corporate HR has historically been a monolith that expects 100% compliance with its systems. But people, as we know, differ emotionally and cognitively. We should be able to use a variety of methods to manage and evaluate people, and help them manage and evaluate themselves. Software delivered as a service is an enabler of that possibility. Here’s a twist: A company won’t have access to the feedback that employees solicit using Rypple. Daniel Debow says that HR folks, well aware of mainstream social software, are ready to embrace this model. I hope he’s right. His favorite recent story about Rypple goes like this: At an HR conference I talked to the CEO of a company that uses Rypple. He’s excited about what we’re doing, but he said: “You have a real problem. Use of your system might make your system obselete. We’ve been using it for a while now, and I’ve noticed that people are much more willing to give me feedback face-to-face, they’re willing to talk to me.” Well that’s the furthest thing from a problem I can imagine. It’s like saying to Facebook, you’ve got a problem, people keep meeting on Facebook and then meeting up in person and creating real relationships offline. Actually that would be problem for Facebook. But Rypple isn’t about pageviews, it’s about helping people improve. Which seems like a great idea to me. You can, by the way, use Rypple not only to solicit anonymized feedback from a chosen set of responders, but also from an open-ended set. So here’s my question: How can I make my ideas more accessible and more actionable? I’m asking a chosen set too, but if you can perceive my blind spot I’d love to know what you see there. On this week’s Innovators show, with Daniel Debow of Rypple, I learned about a cognitive psychological tool called the Johari Window. Rypple focuses on the quadrant of the Johari Window at the intersection of “known to others” and “not known to self” — the so-called blind area. The company is dedicated to the proposition that if we can become more aware of what others know about us that we don’t, we can improve ourselves along various axes: personal, social, and — critically for Rypple’s business model — professionally. How do you gain that awareness? By asking questions like: Am I giving sufficiently clear guidance? or Do I interrupt people too often? You direct these questions to a set of people whose feedback you value. Rypple anonymizes their responses and, to the extent you buy into the service, provides a progressively capable framework within which to continue the dialogue. This is a great idea, and one of the very few appropriate uses for online anonymity that I can imagine. Rypple, as a company, lives at the intersection of a couple of key trends. Social media, obviously, but also the services ecosystem. As we discuss in the podcast, corporate HR has historically been a monolith that expects 100% compliance with its systems. But people, as we know, differ emotionally and cognitively. We should be able to use a variety of methods to manage and evaluate people, and help them manage and evaluate themselves. Software delivered as a service is an enabler of that possibility. Here’s a twist: A company won’t have access to the feedback that employees solicit using Rypple. Daniel Debow says that HR folks, well aware of mainstream social software, are ready to embrace this model. I hope he’s right. His favorite recent story about Rypple goes like this: At an HR conference I talked to the CEO of a company that uses Rypple. He’s excited about what we’re doing, but he said: “You have a real problem. Use of your system might make your system obselete. We’ve been using it for a while now, and I’ve noticed that people are much more willing to give me feedback face-to-face, they’re willing to talk to me.” Well that’s the furthest thing from a problem I can imagine. It’s like saying to Facebook, you’ve got a problem, people keep meeting on Facebook and then meeting up in person and creating real relationships offline. Actually that would be problem for Facebook. But Rypple isn’t about pageviews, it’s about helping people improve. Which seems like a great idea to me. You can, by the way, use Rypple not only to solicit anonymized feedback from a chosen set of responders, but also from an open-ended set. So here’s my question: How can I make my ideas more accessible and more actionable? I’m asking a chosen set too, but if you can perceive my blind spot I’d love to know what you see there. tag:odeo.com,2009-10-13,25279707 Tue, 13 Oct 2009 08:07:03 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25272589-More-visualization-of-Nobel-Peace-Prize-winners-in-Freebase To sharpen the point I made the other day about the eroding bias toward giving the Nobel Peace Prize to Americans and Europeans, here’s a comparison of the nationalities of winners before and after 1960. 1901-2009 nobel peace prize winners by nationality before 1960 after 1960 Here’s another point I forgot to mention. There are gaps in timeline for the Nobel Peace Prize, because it wasn’t awarded in 1914-1918, 1923, 1924, 1928, 1932, 1939-1943, 1948, 1955-1956, 1966-1967 and 1972. The timeline shows those gaps concisely: As in the earlier examples, you can do this with point-and-click filtering in Freebase, no query-writing required. Which is awesome. Finally, Stefano Mazzocchi offers a clarification of a point that came up in our recent interview: I made it sound like Freebase loaded directly IMDB data while what I should have specified is that we loaded the IMDB ‘identifiers’ along with our movie data. Thanks Stefano. And, kudos to the Metaweb team! To sharpen the point I made the other day about the eroding bias toward giving the Nobel Peace Prize to Americans and Europeans, here’s a comparison of the nationalities of winners before and after 1960. 1901-2009 nobel peace prize winners by nationality before 1960 after 1960 Here’s another point I forgot to mention. There are gaps in timeline for the Nobel Peace Prize, because it wasn’t awarded in 1914-1918, 1923, 1924, 1928, 1932, 1939-1943, 1948, 1955-1956, 1966-1967 and 1972. The timeline shows those gaps concisely: As in the earlier examples, you can do this with point-and-click filtering in Freebase, no query-writing required. Which is awesome. Finally, Stefano Mazzocchi offers a clarification of a point that came up in our recent interview: I made it sound like Freebase loaded directly IMDB data while what I should have specified is that we loaded the IMDB ‘identifiers’ along with our movie data. Thanks Stefano. And, kudos to the Metaweb team! To sharpen the point I made the other day about the eroding bias toward giving the Nobel Peace Prize to Americans and Europeans, here’s a comparison of the nationalities of winners before and after 1960. 1901-2009 nobel peace prize winners by nationality before 1960 after 1960 Here’s another point I forgot to mention. There are gaps in timeline for the Nobel Peace Prize, because it wasn’t awarded in 1914-1918, 1923, 1924, 1928, 1932, 1939-1943, 1948, 1955-1956, 1966-1967 and 1972. The timeline shows those gaps concisely: As in the earlier examples, you can do this with point-and-click filtering in Freebase, no query-writing required. Which is awesome. Finally, Stefano Mazzocchi offers a clarification of a point that came up in our recent interview: I made it sound like Freebase loaded directly IMDB data while what I should have specified is that we loaded the IMDB ‘identifiers’ along with our movie data. Thanks Stefano. And, kudos to the Metaweb team! tag:odeo.com,2009-10-12,25272589 Mon, 12 Oct 2009 05:50:57 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25269103-Recovering-forgotten-methods-of-construction After feasting on audio podcasts for years, I realized that I don’t always want somebody else’s voice in my head while running, biking, and hiking. So I went on an audio fast for a couple of months. But now I’m ready for more input, and I’m once again reminded how wonderful it is to be able to bring engaging minds with me on my outdoor excursions. One of my companions on yesterday’s hike was John Ochsendorf, a historian and structural engineer who explores the relevance of ancient and sometimes forgotten construction methods, like Incan suspension bridges woven from grass. One of his passions is Guastavino tile vaulting, a system that was patented in 1885. Although widely used in many notable structures — including Grand Central Station — Ochsendorf says that some of these structures have been torn down and rebuilt conventionally because modern engineers no longer understand how the Guastavino system works, and cannot evaluate its integrity.... After feasting on audio podcasts for years, I realized that I don’t always want somebody else’s voice in my head while running, biking, and hiking. So I went on an audio fast for a couple of months. But now I’m ready for more input, and I’m once again reminded how wonderful it is to be able to bring engaging minds with me on my outdoor excursions. One of my companions on yesterday’s hike was John Ochsendorf, a historian and structural engineer who explores the relevance of ancient and sometimes forgotten construction methods, like Incan suspension bridges woven from grass. One of his passions is Guastavino tile vaulting, a system that was patented in 1885. Although widely used in many notable structures — including Grand Central Station — Ochsendorf says that some of these structures have been torn down and rebuilt conventionally because modern engineers no longer understand how the Guastavino system works, and cannot evaluate its integrity. This theme of forgotten knowledge echoes something I heard in Amory Lovins’ epic MAP/Ming lecture series. He describes a large government building in Washington, DC, that was made of stone and cooled by a carefully-designed pattern of air flow. The cooling system wasn’t completely passive, though. You had to open and close windows in a particular sequence throughout the day. Now that building is cooled by hundreds of window-mounted air conditioners. I’m sure our modernn expectation of extreme cooling is part of the reason why. But Lovins also says that air conditioning became necessary because people forgot how to operate the building. I love the idea of recovering — and scientifically validating — forgotten knowledge. That’s what John Ochsendorf’s research group does. One of his students, Joe Dahmen, did a project called Rammed Earth — a long-term experiment to see if that ancient construction method could actually work in present-day New England. John Ochsendorf says: Historical methods of construction that are very green, very local, may create beautiful low-energy architecture, we’ve forgotten how to do them. So we have to rediscover them, and do testing to prove to clients and building owners that you can use these methods. And it’s a good example of MIT’s motto of mind and hand. We don’t like to just read about rammed earth walls, we like to get dirty and build them. Very cool. I think the MacArthur Foundation invested wisely in this guy. After feasting on audio podcasts for years, I realized that I don’t always want somebody else’s voice in my head while running, biking, and hiking. So I went on an audio fast for a couple of months. But now I’m ready for more input, and I’m once again reminded how wonderful it is to be able to bring engaging minds with me on my outdoor excursions. One of my companions on yesterday’s hike was John Ochsendorf, a historian and structural engineer who explores the relevance of ancient and sometimes forgotten construction methods, like Incan suspension bridges woven from grass. One of his passions is Guastavino tile vaulting, a system that was patented in 1885. Although widely used in many notable structures — including Grand Central Station — Ochsendorf says that some of these structures have been torn down and rebuilt conventionally because modern engineers no longer understand how the Guastavino system works, and cannot evaluate its integrity. This theme of forgotten knowledge echoes something I heard in Amory Lovins’ epic MAP/Ming lecture series. He describes a large government building in Washington, DC, that was made of stone and cooled by a carefully-designed pattern of air flow. The cooling system wasn’t completely passive, though. You had to open and close windows in a particular sequence throughout the day. Now that building is cooled by hundreds of window-mounted air conditioners. I’m sure our modernn expectation of extreme cooling is part of the reason why. But Lovins also says that air conditioning became necessary because people forgot how to operate the building. I love the idea of recovering — and scientifically validating — forgotten knowledge. That’s what John Ochsendorf’s research group does. One of his students, Joe Dahmen, did a project called Rammed Earth — a long-term experiment to see if that ancient construction method could actually work in present-day New England. John Ochsendorf says: Historical methods of construction that are very green, very local, may create beautiful low-energy architecture, we’ve forgotten how to do them. So we have to rediscover them, and do testing to prove to clients and building owners that you can use these methods. And it’s a good example of MIT’s motto of mind and hand. We don’t like to just read about rammed earth walls, we like to get dirty and build them. Very cool. I think the MacArthur Foundation invested wisely in this guy. tag:odeo.com,2009-10-11,25269103 Sun, 11 Oct 2009 09:46:22 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25264174-Visualizing-Nobel-Peace-Prize-winners-in-Freebase When I watched Barack Obama accept the Nobel Peace Prize, I thought about how the world has changed since the inception of the prize, and how it will continue to change. Since the winners of the Prize are themselves a reflection of what’s changing, I thought I’d try using Freebase to visualize them over the century the Prize has existed. What you can find out, with Freebase, depends on its coverage of the topics you’re asking about. So realize that what I’ll show here is possible because Nobel Peace Prize winners are a well-covered topic. Still, it’s wildly impressive. The Nobel site tells us that 89 Nobel Peace Prizes have been awarded since 1901. I haven’t been able to reproduce that number in Freebase because there are multiple winners in a few years, and I haven’t found a way group results by year. But for my purposes this related query is good enough: That number, 100, isn’t as closely related to 89 as you might think. It’s ... When I watched Barack Obama accept the Nobel Peace Prize, I thought about how the world has changed since the inception of the prize, and how it will continue to change. Since the winners of the Prize are themselves a reflection of what’s changing, I thought I’d try using Freebase to visualize them over the century the Prize has existed. What you can find out, with Freebase, depends on its coverage of the topics you’re asking about. So realize that what I’ll show here is possible because Nobel Peace Prize winners are a well-covered topic. Still, it’s wildly impressive. The Nobel site tells us that 89 Nobel Peace Prizes have been awarded since 1901. I haven’t been able to reproduce that number in Freebase because there are multiple winners in a few years, and I haven’t found a way group results by year. But for my purposes this related query is good enough: That number, 100, isn’t as closely related to 89 as you might think. It’s less by the number of years no award was given, but more by the number of recipients in multiple-award years. Perhaps a Freebase guru can show us how to measure those uncertainties, but I’ve eyeballed them and I don’t think they invalidate my results. How did I wind up querying the topic /award/award_winner? It wasn’t immediately obvious. I spent a while searching and then exploring the facets that emerged, including: http://www.freebase.com/view/en/nobel_prize http://www.freebase.com/view/award/award_category http://www.freebase.com/view/award/award_honor http://www.freebase.com/view/award/award_winner The crazy thing about Freebase is that, in a way, it doesn’t matter where you start. Everything’s connected to everything, so you can pick up any node of the graph and re-dangle the rest. Except when you can’t. I haven’t yet gotten a good feel for which paths to prefer and why. But in the end I came up with the kind of results I’d envisioned: 1901-2009 nobel peace prize winners by gender male female 1901-2009 nobel peace prize winners by nationality male female Taken together they show a couple of trends. First, of course, we see most female winners after about 1960. Second, we see a more even geographic distribution of female winners because, prior to 1960, most winners were not only male but also American or European. These results didn’t surprise me. What did is the relative ease with which I was able to discover and document them. I thought it would be necessary to write MQL queries in order to do this kind of analysis. I’d previously done a bit of work with MQL, and dug further into it this time around. But in the end I found that it was just as effective to use interactive filtering. Now to be clear, getting the software to actually do the things I’ve shown here wasn’t a cakewalk. I had to develop a feel for the web of topics in the domain I chose. And it’s painfully slow to add and drop filters. But still, it’s doable. And you can do it yourself by pointing and clicking. That is an astonishing tour de force, and a glimpse of what things will be like when we can all fluently visualize information about our world. When I watched Barack Obama accept the Nobel Peace Prize, I thought about how the world has changed since the inception of the prize, and how it will continue to change. Since the winners of the Prize are themselves a reflection of what’s changing, I thought I’d try using Freebase to visualize them over the century the Prize has existed. What you can find out, with Freebase, depends on its coverage of the topics you’re asking about. So realize that what I’ll show here is possible because Nobel Peace Prize winners are a well-covered topic. Still, it’s wildly impressive. The Nobel site tells us that 89 Nobel Peace Prizes have been awarded since 1901. I haven’t been able to reproduce that number in Freebase because there are multiple winners in a few years, and I haven’t found a way group results by year. But for my purposes this related query is good enough: That number, 100, isn’t as closely related to 89 as you might think. It’s less by the number of years no award was given, but more by the number of recipients in multiple-award years. Perhaps a Freebase guru can show us how to measure those uncertainties, but I’ve eyeballed them and I don’t think they invalidate my results. How did I wind up querying the topic /award/award_winner? It wasn’t immediately obvious. I spent a while searching and then exploring the facets that emerged, including: http://www.freebase.com/view/en/nobel_prize http://www.freebase.com/view/award/award_category http://www.freebase.com/view/award/award_honor http://www.freebase.com/view/award/award_winner The crazy thing about Freebase is that, in a way, it doesn’t matter where you start. Everything’s connected to everything, so you can pick up any node of the graph and re-dangle the rest. Except when you can’t. I haven’t yet gotten a good feel for which paths to prefer and why. But in the end I came up with the kind of results I’d envisioned: 1901-2009 nobel peace prize winners by gender male female 1901-2009 nobel peace prize winners by nationality male female Taken together they show a couple of trends. First, of course, we see most female winners after about 1960. Second, we see a more even geographic distribution of female winners because, prior to 1960, most winners were not only male but also American or European. These results didn’t surprise me. What did is the relative ease with which I was able to discover and document them. I thought it would be necessary to write MQL queries in order to do this kind of analysis. I’d previously done a bit of work with MQL, and dug further into it this time around. But in the end I found that it was just as effective to use interactive filtering. Now to be clear, getting the software to actually do the things I’ve shown here wasn’t a cakewalk. I had to develop a feel for the web of topics in the domain I chose. And it’s painfully slow to add and drop filters. But still, it’s doable. And you can do it yourself by pointing and clicking. That is an astonishing tour de force, and a glimpse of what things will be like when we can all fluently visualize information about our world. tag:odeo.com,2009-10-10,25264174 Sat, 10 Oct 2009 06:47:05 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25254764-Magic-glasses-and-magic-projectors-Private-versus-public-augmentation-of-experience At its core, your browser is powered by an engine called the Document Object Model, hereafter DOM. You can think of the DOM as an outline, and the browser as an outline processor that shows and hides things, displays things in different ways, and even adds, removes, or rearranges things. Nowadays what you see, when you view a web page, is the result of a complex interaction between data and code. The data is the HTML content of the page, and the code is its JavaScript behavior. But these are slippery terms. A lot of content never originates as HTML, but is instead produced dynamically — by a web server, but also quite possibly in the browser as it manipulates the DOM. And a lot of behavior happens opportunistically in response to content on the page. This arrangement has radical implications. For example, back in 2002 I invented LibraryLookup, a bookmarklet that noticed when you were visiting an Amazon or Barnes and Noble book page and offered a one-click search for that book ... At its core, your browser is powered by an engine called the Document Object Model, hereafter DOM. You can think of the DOM as an outline, and the browser as an outline processor that shows and hides things, displays things in different ways, and even adds, removes, or rearranges things. Nowadays what you see, when you view a web page, is the result of a complex interaction between data and code. The data is the HTML content of the page, and the code is its JavaScript behavior. But these are slippery terms. A lot of content never originates as HTML, but is instead produced dynamically — by a web server, but also quite possibly in the browser as it manipulates the DOM. And a lot of behavior happens opportunistically in response to content on the page. This arrangement has radical implications. For example, back in 2002 I invented LibraryLookup, a bookmarklet that noticed when you were visiting an Amazon or Barnes and Noble book page and offered a one-click search for that book in your local library. A few years later, a Firefox extension called Greasemonkey arrived on the scene. It offered two capabilities that, working together, enabled a zero-click LibraryLookup. First, it could call out to a web service. Second, it could modify the DOM based on the response. Putting these two things together, I wrote a script that would notice that you were visiting an Amazon book page, check to see if the book was available at your local library, and if so, insert a paragraph into the DOM that said: “Hey, it’s available at the [YOUR LIBRARY NAME] library!” Is this kosher? I think so, but it’s a tricky question. At the time I made a short screencast that reflected on questions of ownership and fair use in an environment that’s designed and built to support intermediation and remixing. These questions were still largely hypothetical, though, because Firefox users who had also installed Greasemonkey were a very small number indeed. But now, thanks to modern browser-independent JavaScript libraries like jQuery, those hypothetical questions are becoming very real. Here’s Phil Windley demonstrating his 2009 version of LibraryLookup: The example comes from Phil’s recent essay The Forgotten Edge: Building a Purpose-Centric Web, which makes the case for contextualized browsing as enabled by libraries like jQuery and by infrastructure like that provided by Phil’s company, Kynetx. In Phil’s next blog item, Claiming My Right to a Purpose-Centric Web: SideWiki, he asserts: I claim the right to mash-up, remix, annotate, augment, and otherwise modify Web content for my purposes in my browser using any tool I choose and I extend to everyone else that same privilege. That item grew a long tail of comments. It includes some interesting back-and-forth between Phil Windley and Dave Winer, but I want to focus on this observation from Greg Yardley: Sites also generally come with a contract attached – some implicit (the view-through), some explicit (the click-through) – and these contracts, done correctly, are generally enforceable. This whole post mystifies me, because you don’t have the the right to mash-up, remix, annotate, augment, and otherwise modify Web content – it’s not your content. Earlier in the thread, Jeremy Pickens cited an example of such a contract: Google’s terms of service: 8.2 You should be aware that Content presented to you as part of the Services, including but not limited to advertisements in the Services and sponsored Content within the Services may be protected by intellectual property rights which are owned by the sponsors or advertisers who provide that Content to Google (or by other persons or companies on their behalf). You may not modify, rent, lease, loan, sell, distribute or create derivative works based on this Content (either in whole or in part) unless you have been specifically told that you may do so by Google or by the owners of that Content, in a separate agreement. In response to Greg Yardley, Phil Rees cites fair use: Actually we do have those rights. http://www.law.cornell.edu/uscode/17/107.html I believe so too. Sooner or later, that belief will be tested. After my March interview with Phil about Kynetx, I wrote: There’s a continuum of ways in which I can modify a web page in a browser, ranging from font enlargement to translation to contexual overlays. I wouldn’t draw a line anywhere along that continuum. It seems to me that I’m entitled to view the world through any lens I choose. This doesn’t only apply to my view of the virtual world, by the way. It will apply to my view of the physical world too. We don’t yet have magic glasses that overlay web prices on shelf items, or web reputations on store signage, but someday we will. I can’t see how I could be prevented from creating a heads-up display — for realspace or cyberspace — that’s advantageous to me. But I’ve got a hunch that those magic glasses are going to be controversial. I wonder if it’s going to boil down to magic glasses versus magic projectors. Or, in other words, private versus public augmentation of our experiences of the virtual and real worlds. I can wear my magic glasses, but I can’t necessarily project the view that I’m seeing. At its core, your browser is powered by an engine called the Document Object Model, hereafter DOM. You can think of the DOM as an outline, and the browser as an outline processor that shows and hides things, displays things in different ways, and even adds, removes, or rearranges things. Nowadays what you see, when you view a web page, is the result of a complex interaction between data and code. The data is the HTML content of the page, and the code is its JavaScript behavior. But these are slippery terms. A lot of content never originates as HTML, but is instead produced dynamically — by a web server, but also quite possibly in the browser as it manipulates the DOM. And a lot of behavior happens opportunistically in response to content on the page. This arrangement has radical implications. For example, back in 2002 I invented LibraryLookup, a bookmarklet that noticed when you were visiting an Amazon or Barnes and Noble book page and offered a one-click search for that book in your local library. A few years later, a Firefox extension called Greasemonkey arrived on the scene. It offered two capabilities that, working together, enabled a zero-click LibraryLookup. First, it could call out to a web service. Second, it could modify the DOM based on the response. Putting these two things together, I wrote a script that would notice that you were visiting an Amazon book page, check to see if the book was available at your local library, and if so, insert a paragraph into the DOM that said: “Hey, it’s available at the [YOUR LIBRARY NAME] library!” Is this kosher? I think so, but it’s a tricky question. At the time I made a short screencast that reflected on questions of ownership and fair use in an environment that’s designed and built to support intermediation and remixing. These questions were still largely hypothetical, though, because Firefox users who had also installed Greasemonkey were a very small number indeed. But now, thanks to modern browser-independent JavaScript libraries like jQuery, those hypothetical questions are becoming very real. Here’s Phil Windley demonstrating his 2009 version of LibraryLookup: The example comes from Phil’s recent essay The Forgotten Edge: Building a Purpose-Centric Web, which makes the case for contextualized browsing as enabled by libraries like jQuery and by infrastructure like that provided by Phil’s company, Kynetx. In Phil’s next blog item, Claiming My Right to a Purpose-Centric Web: SideWiki, he asserts: I claim the right to mash-up, remix, annotate, augment, and otherwise modify Web content for my purposes in my browser using any tool I choose and I extend to everyone else that same privilege. That item grew a long tail of comments. It includes some interesting back-and-forth between Phil Windley and Dave Winer, but I want to focus on this observation from Greg Yardley: Sites also generally come with a contract attached – some implicit (the view-through), some explicit (the click-through) – and these contracts, done correctly, are generally enforceable. This whole post mystifies me, because you don’t have the the right to mash-up, remix, annotate, augment, and otherwise modify Web content – it’s not your content. Earlier in the thread, Jeremy Pickens cited an example of such a contract: Google’s terms of service: 8.2 You should be aware that Content presented to you as part of the Services, including but not limited to advertisements in the Services and sponsored Content within the Services may be protected by intellectual property rights which are owned by the sponsors or advertisers who provide that Content to Google (or by other persons or companies on their behalf). You may not modify, rent, lease, loan, sell, distribute or create derivative works based on this Content (either in whole or in part) unless you have been specifically told that you may do so by Google or by the owners of that Content, in a separate agreement. In response to Greg Yardley, Phil Rees cites fair use: Actually we do have those rights. http://www.law.cornell.edu/uscode/17/107.html I believe so too. Sooner or later, that belief will be tested. After my March interview with Phil about Kynetx, I wrote: There’s a continuum of ways in which I can modify a web page in a browser, ranging from font enlargement to translation to contexual overlays. I wouldn’t draw a line anywhere along that continuum. It seems to me that I’m entitled to view the world through any lens I choose. This doesn’t only apply to my view of the virtual world, by the way. It will apply to my view of the physical world too. We don’t yet have magic glasses that overlay web prices on shelf items, or web reputations on store signage, but someday we will. I can’t see how I could be prevented from creating a heads-up display — for realspace or cyberspace — that’s advantageous to me. But I’ve got a hunch that those magic glasses are going to be controversial. I wonder if it’s going to boil down to magic glasses versus magic projectors. Or, in other words, private versus public augmentation of our experiences of the virtual and real worlds. I can wear my magic glasses, but I can’t necessarily project the view that I’m seeing. tag:odeo.com,2009-10-08,25254764 Thu, 08 Oct 2009 08:01:57 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25248994-Talking-with-Victoria-Stodden-about-Science-Commons On this week’s Innovators show I spoke with Victoria Stodden about Science Commons, an effort to bring the values and methods of Creative Commons to the realm of science. Because modern science is so data- and computation-intensive, Science Commons provides legal tools that govern the sharing of data and code. There are lots of good reasons to share the artifacts of scientific computation. Victoria particularly focuses on the benefit of reproducibility. It’s one thing to say that your analysis of a data set leads to a conclusion. It’s quite another to give me your data, and the code you used to process it, and invite me to repeat the experiment. In this kind of discussion, the word “repository” always comes up. If you put your stuff into a repository, I can take it out and work with it. But I’ve always had a bit of an allergic reaction to that word, and during this podcast I realized why: it connotes a burial ground. What goes into a repository ju... On this week’s Innovators show I spoke with Victoria Stodden about Science Commons, an effort to bring the values and methods of Creative Commons to the realm of science. Because modern science is so data- and computation-intensive, Science Commons provides legal tools that govern the sharing of data and code. There are lots of good reasons to share the artifacts of scientific computation. Victoria particularly focuses on the benefit of reproducibility. It’s one thing to say that your analysis of a data set leads to a conclusion. It’s quite another to give me your data, and the code you used to process it, and invite me to repeat the experiment. In this kind of discussion, the word “repository” always comes up. If you put your stuff into a repository, I can take it out and work with it. But I’ve always had a bit of an allergic reaction to that word, and during this podcast I realized why: it connotes a burial ground. What goes into a repository just sits there. It might be looked at, it might be copied, but it’s essentially inert, a dead artifact divorced from its live context. Sooner or later, cloud computing will change that. The live context in which primary research happens will be a shareable online space. Publishing won’t entail pushing your code and data to a repository, but rather granting access to that space. It’s a hard conceptual shift to make, though. We think of publishing as a way of pushing stuff out from where we work on it to someplace else where people can get at it. But when we do our work in the cloud, publishing is really just an invitation to visit us there. On this week’s Innovators show I spoke with Victoria Stodden about Science Commons, an effort to bring the values and methods of Creative Commons to the realm of science. Because modern science is so data- and computation-intensive, Science Commons provides legal tools that govern the sharing of data and code. There are lots of good reasons to share the artifacts of scientific computation. Victoria particularly focuses on the benefit of reproducibility. It’s one thing to say that your analysis of a data set leads to a conclusion. It’s quite another to give me your data, and the code you used to process it, and invite me to repeat the experiment. In this kind of discussion, the word “repository” always comes up. If you put your stuff into a repository, I can take it out and work with it. But I’ve always had a bit of an allergic reaction to that word, and during this podcast I realized why: it connotes a burial ground. What goes into a repository just sits there. It might be looked at, it might be copied, but it’s essentially inert, a dead artifact divorced from its live context. Sooner or later, cloud computing will change that. The live context in which primary research happens will be a shareable online space. Publishing won’t entail pushing your code and data to a repository, but rather granting access to that space. It’s a hard conceptual shift to make, though. We think of publishing as a way of pushing stuff out from where we work on it to someplace else where people can get at it. But when we do our work in the cloud, publishing is really just an invitation to visit us there. tag:odeo.com,2009-10-07,25248994 Wed, 07 Oct 2009 08:59:07 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25203619-Querying-mobile-data-objects-with-LINQ I’m using US census data to look up the estimated populations of the cities and towns running elmcity hubs. The dataset is just plain old CSV (comma-separated variable), a format that’s more popular than ever thanks in part to a new wave of web-based data services like DabbleDB, ManyEyes, and others. For my purposes, simple pattern matching was enough to look up the population of a city and state. But I’d been meaning to try out LINQtoCSV, the .NET equivalent of my old friend, Python’s csv module. As happens lately, I was struck by the convergence of the languages. Here’s a side-by-side comparison of Python and C# using their respective CSV modules to query for the population of Keene, NH: Python C#     i_name = 5 i_statename = 6   i_pop2008 = 17     handle = urllib.urlopen(url)                 reader = csv.reader(handle, delimiter=',')     rows = itertools.ifilter(lambda x : &nb... I’m using US census data to look up the estimated populations of the cities and towns running elmcity hubs. The dataset is just plain old CSV (comma-separated variable), a format that’s more popular than ever thanks in part to a new wave of web-based data services like DabbleDB, ManyEyes, and others. For my purposes, simple pattern matching was enough to look up the population of a city and state. But I’d been meaning to try out LINQtoCSV, the .NET equivalent of my old friend, Python’s csv module. As happens lately, I was struck by the convergence of the languages. Here’s a side-by-side comparison of Python and C# using their respective CSV modules to query for the population of Keene, NH: Python C#     i_name = 5 i_statename = 6   i_pop2008 = 17     handle = urllib.urlopen(url)                 reader = csv.reader(handle, delimiter=',')     rows = itertools.ifilter(lambda x :   x[i_name].startswith('Keene') and   x[i_statename] == 'New Hampshire', reader)   found_rows = list(rows)       count = len(found_rows)   if ( count > 0 ): pop = int ( found_rows[0][i_pop2008] ) public class USCensusPopulationData { public string NAME; public string STATENAME; ... etc. ... public string POP_2008; } var csv = new WebClient(). DownloadString(url); var stream = new MemoryStream( Encoding.UTF8.GetBytes(csv)); var sr = new StreamReader(stream); var cc = new CsvContext(); var fd = new CsvFileDescription { }; var reader = cc.Read<USCensusPopulationData>(sr, fd); var rows = reader.ToList(); var found_rows = rows.FindAll(row => row.name.StartsWith('Keene') && row.statename == 'New Hampshire'); var count = rows.Count; if ( count > 0 ) pop = Convert.ToInt32( found_rows[0].pop_2008) Things don’t line up quite as neatly as in my earlier example, or as in the A/B comparison (from way back in 2005) between my first LINQ example and Sam Ruby’s Ruby equivalent. But the two examples share a common approach based on iterators and filters. This idea of running queries over simple text files is something I first ran into long ago in the form of the ODBC Text driver, which provides SQL queries over comma-separated data. I’ve always loved this style of data access, and it remains incredibly handy. Yes, some data sets are huge. But the 80,000 rows of that census file add up to only 8MB file. The file isn’t growing quickly, and it can tell a lot of stories. Here’s one: 2000 - 2008 population loss in NH -8.09% Berlin city -3.67% Coos County -1.85% Portsmouth city -1.85% Plaistow town -1.78% Balance of Coos County -1.43% Claremont city -1.02% Lancaster town -0.99% Rye town -0.81% Keene city -0.23% Nashua city In both Python and C# you can work directly with the iterators returned by the CSV modules to accomplish this kind of query. Here’s a Python version: import urllib, itertools, csv i_name = 5 i_statename = 6 i_pop2000 = 9 i_pop2008 = 17 def make_reader(): handle = open('pop.csv') return csv.reader(handle, delimiter=',') def unique(rows): dict = {} for row in rows: key = "%s %s %s %s" % (i_name, i_statename, row[i_pop2000], row[i_pop2008]) dict[key] = row list = [] for key in dict: list.append( dict[key] ) return list def percent(row,a,b): pct = - ( float(row[a]) / float(row[b]) * 100 - 100 ) return pct def change(x,state,minpop=1): statename = x[i_statename] p2000 = int(x[i_pop2000]) p2008 = int(x[i_pop2008]) return ( statename==state and p2008 > minpop and p2008 < p2000 ) state = 'New Hampshire' reader = make_reader() reader.next() # skip fieldnames rows = itertools.ifilter(lambda x : change(x,state,minpop=3000), reader) l = list(rows) l = unique(l) l.sort(lambda x,y: cmp(percent(x,i_pop2000,i_pop2008), percent(y,i_pop2000,i_pop2008))) for row in l: print "%2.2f%% %s" % ( percent(row,i_pop2000,i_pop2008), row[i_name] ) A literal C# translation could do all the same things in the same ways: Convert the iterator into a list, use a dictionary to remove duplication, filter the list with a lambda function, sort the list with another lambda function. As queries grow more complex, though, you tend to want a more declarative style. To do that in Python, you’d likely import the CSV file into a SQL database — perhaps SQLite in order to stay true to the lightweight nature of this example. Then you’d ship queries to the database in the form of SQL statements. But you’re crossing a chasm when you do that. The database’s type system isn’t the same as Python’s. And database’s internal language for writing functions won’t be Python either. In the case of SQLite, there won’t even be an internal language. With LINQ there’s no chasm to cross. Here’s the LINQ code that produces the same result: var census_rows = make_reader(); var distinct_rows = census_rows.Distinct(new CensusRowComparer()); var threshold = 3000; var rows = from row in distinct_rows where row.STATENAME == statename && Convert.ToInt32(row.POP_2008) > threshold && Convert.ToInt32(row.POP_2008) < Convert.ToInt32(row.POP_2000) orderby percent(row.POP_2000,row.POP_2008) select new { name = row.NAME, pop2000 = row.POP_2000, pop2008 = row.POP_2008 }; foreach (var row in rows) Console.WriteLine("{0:0.00}% {1}", percent(row.pop2000,row.pop2008), row.name ); You can see the supporting pieces below. There are a number of aspects to this approach that I’m enjoying. It’s useful, for example, that every row of data becomes an object whose properties are available to the editor and the debugger. But what really delights me is the way that the query context and the results context share the same environment, just as in the Python example above. In this (slightly contrived) example I’m using the percent function in both contexts. With LINQ to CSV I’m now using four flavors of LINQ in my project. Two are built into the .NET Framework: LINQ to XML, and LINQ to native .NET objects. And two are extensions: LINQ to CSV, and LINQ to JSON. In all four cases, I’m querying some kind of mobile data object: an RSS feed, a binary .NET object retrieved from the Azure blob store, a JSON response, and now a CSV file. Six years ago I was part of a delegation from InfoWorld that visited Microsoft for a preview technologies in the pipeline. At a dinner I sat with Anders Hejslberg and listened to him lay out his vision for what would become LINQ. There were two key goals. First, a single environment for query and results. Second, a common approach to many flavors of data. I think he nailed both pretty well. And it’s timely because the cloud isn’t just an ecosystem of services, it’s also an ecosystem of mobile data objects that come in a variety of flavors. private static float percent(string a, string b) { var y0 = float.Parse(a); var y1 = float.Parse(b); return - ( y0 / y1 * 100 - 100); } private static IEnumerable<USCensusPopulationData> make_reader() { var h = new FileStream("pop.csv", FileMode.Open); var bytes = new byte[h.Length]; h.Read(bytes, 0, (Int32)h.Length); bytes = Encoding.UTF8.GetBytes(Encoding.UTF8.GetString(bytes)); var stream = new MemoryStream(bytes); var sr = new StreamReader(stream); var cc = new CsvContext(); var fd = new CsvFileDescription { }; var census_rows = cc.Read<USCensusPopulationData>(sr, fd); return census_rows; } public class USCensusPopulationData { public string SUMLEV; public string state; public string county; public string PLACE; public string cousub; public string NAME; public string STATENAME; public string POPCENSUS_2000; public string POPBASE_2000; public string POP_2000; public string POP_2001; public string POP_2002; public string POP_2003; public string POP_2004; public string POP_2005; public string POP_2006; public string POP_2007; public string POP_2008; public override string ToString() { return NAME + ", " + STATENAME + " " + "pop2000=" + POP_2000 + " | " + "pop2008=" + POP_2008; } } public class CensusRowComparer : IEqualityComparer<USCensusPopulationData> { public bool Equals(USCensusPopulationData x, USCensusPopulationData y) { return x.NAME == y.NAME && x.STATENAME == y.STATENAME ; } public int GetHashCode(USCensusPopulationData obj) { var hash = obj.ToString(); return hash.GetHashCode(); } } I’m using US census data to look up the estimated populations of the cities and towns running elmcity hubs. The dataset is just plain old CSV (comma-separated variable), a format that’s more popular than ever thanks in part to a new wave of web-based data services like DabbleDB, ManyEyes, and others. For my purposes, simple pattern matching was enough to look up the population of a city and state. But I’d been meaning to try out LINQtoCSV, the .NET equivalent of my old friend, Python’s csv module. As happens lately, I was struck by the convergence of the languages. Here’s a side-by-side comparison of Python and C# using their respective CSV modules to query for the population of Keene, NH: Python C#     i_name = 5 i_statename = 6   i_pop2008 = 17     handle = urllib.urlopen(url)                 reader = csv.reader(handle, delimiter=',')     rows = itertools.ifilter(lambda x :   x[i_name].startswith('Keene') and   x[i_statename] == 'New Hampshire', reader)   found_rows = list(rows)       count = len(found_rows)   if ( count > 0 ): pop = int ( found_rows[0][i_pop2008] ) public class USCensusPopulationData { public string NAME; public string STATENAME; ... etc. ... public string POP_2008; } var csv = new WebClient(). DownloadString(url); var stream = new MemoryStream( Encoding.UTF8.GetBytes(csv)); var sr = new StreamReader(stream); var cc = new CsvContext(); var fd = new CsvFileDescription { }; var reader = cc.Read<USCensusPopulationData>(sr, fd); var rows = reader.ToList(); var found_rows = rows.FindAll(row => row.name.StartsWith('Keene') && row.statename == 'New Hampshire'); var count = rows.Count; if ( count > 0 ) pop = Convert.ToInt32( found_rows[0].pop_2008) Things don’t line up quite as neatly as in my earlier example, or as in the A/B comparison (from way back in 2005) between my first LINQ example and Sam Ruby’s Ruby equivalent. But the two examples share a common approach based on iterators and filters. This idea of running queries over simple text files is something I first ran into long ago in the form of the ODBC Text driver, which provides SQL queries over comma-separated data. I’ve always loved this style of data access, and it remains incredibly handy. Yes, some data sets are huge. But the 80,000 rows of that census file add up to only 8MB file. The file isn’t growing quickly, and it can tell a lot of stories. Here’s one: 2000 - 2008 population loss in NH -8.09% Berlin city -3.67% Coos County -1.85% Portsmouth city -1.85% Plaistow town -1.78% Balance of Coos County -1.43% Claremont city -1.02% Lancaster town -0.99% Rye town -0.81% Keene city -0.23% Nashua city In both Python and C# you can work directly with the iterators returned by the CSV modules to accomplish this kind of query. Here’s a Python version: import urllib, itertools, csv i_name = 5 i_statename = 6 i_pop2000 = 9 i_pop2008 = 17 def make_reader(): handle = open('pop.csv') return csv.reader(handle, delimiter=',') def unique(rows): dict = {} for row in rows: key = "%s %s %s %s" % (i_name, i_statename, row[i_pop2000], row[i_pop2008]) dict[key] = row list = [] for key in dict: list.append( dict[key] ) return list def percent(row,a,b): pct = - ( float(row[a]) / float(row[b]) * 100 - 100 ) return pct def change(x,state,minpop=1): statename = x[i_statename] p2000 = int(x[i_pop2000]) p2008 = int(x[i_pop2008]) return ( statename==state and p2008 > minpop and p2008 < p2000 ) state = 'New Hampshire' reader = make_reader() reader.next() # skip fieldnames rows = itertools.ifilter(lambda x : change(x,state,minpop=3000), reader) l = list(rows) l = unique(l) l.sort(lambda x,y: cmp(percent(x,i_pop2000,i_pop2008), percent(y,i_pop2000,i_pop2008))) for row in l: print "%2.2f%% %s" % ( percent(row,i_pop2000,i_pop2008), row[i_name] ) A literal C# translation could do all the same things in the same ways: Convert the iterator into a list, use a dictionary to remove duplication, filter the list with a lambda function, sort the list with another lambda function. As queries grow more complex, though, you tend to want a more declarative style. To do that in Python, you’d likely import the CSV file into a SQL database — perhaps SQLite in order to stay true to the lightweight nature of this example. Then you’d ship queries to the database in the form of SQL statements. But you’re crossing a chasm when you do that. The database’s type system isn’t the same as Python’s. And database’s internal language for writing functions won’t be Python either. In the case of SQLite, there won’t even be an internal language. With LINQ there’s no chasm to cross. Here’s the LINQ code that produces the same result: var census_rows = make_reader(); var distinct_rows = census_rows.Distinct(new CensusRowComparer()); var threshold = 3000; var rows = from row in distinct_rows where row.STATENAME == statename && Convert.ToInt32(row.POP_2008) > threshold && Convert.ToInt32(row.POP_2008) < Convert.ToInt32(row.POP_2000) orderby percent(row.POP_2000,row.POP_2008) select new { name = row.NAME, pop2000 = row.POP_2000, pop2008 = row.POP_2008 }; foreach (var row in rows) Console.WriteLine("{0:0.00}% {1}", percent(row.pop2000,row.pop2008), row.name ); You can see the supporting pieces below. There are a number of aspects to this approach that I’m enjoying. It’s useful, for example, that every row of data becomes an object whose properties are available to the editor and the debugger. But what really delights me is the way that the query context and the results context share the same environment, just as in the Python example above. In this (slightly contrived) example I’m using the percent function in both contexts. With LINQ to CSV I’m now using four flavors of LINQ in my project. Two are built into the .NET Framework: LINQ to XML, and LINQ to native .NET objects. And two are extensions: LINQ to CSV, and LINQ to JSON. In all four cases, I’m querying some kind of mobile data object: an RSS feed, a binary .NET object retrieved from the Azure blob store, a JSON response, and now a CSV file. Six years ago I was part of a delegation from InfoWorld that visited Microsoft for a preview technologies in the pipeline. At a dinner I sat with Anders Hejslberg and listened to him lay out his vision for what would become LINQ. There were two key goals. First, a single environment for query and results. Second, a common approach to many flavors of data. I think he nailed both pretty well. And it’s timely because the cloud isn’t just an ecosystem of services, it’s also an ecosystem of mobile data objects that come in a variety of flavors. private static float percent(string a, string b) { var y0 = float.Parse(a); var y1 = float.Parse(b); return - ( y0 / y1 * 100 - 100); } private static IEnumerable<USCensusPopulationData> make_reader() { var h = new FileStream("pop.csv", FileMode.Open); var bytes = new byte[h.Length]; h.Read(bytes, 0, (Int32)h.Length); bytes = Encoding.UTF8.GetBytes(Encoding.UTF8.GetString(bytes)); var stream = new MemoryStream(bytes); var sr = new StreamReader(stream); var cc = new CsvContext(); var fd = new CsvFileDescription { }; var census_rows = cc.Read<USCensusPopulationData>(sr, fd); return census_rows; } public class USCensusPopulationData { public string SUMLEV; public string state; public string county; public string PLACE; public string cousub; public string NAME; public string STATENAME; public string POPCENSUS_2000; public string POPBASE_2000; public string POP_2000; public string POP_2001; public string POP_2002; public string POP_2003; public string POP_2004; public string POP_2005; public string POP_2006; public string POP_2007; public string POP_2008; public override string ToString() { return NAME + ", " + STATENAME + " " + "pop2000=" + POP_2000 + " | " + "pop2008=" + POP_2008; } } public class CensusRowComparer : IEqualityComparer<USCensusPopulationData> { public bool Equals(USCensusPopulationData x, USCensusPopulationData y) { return x.NAME == y.NAME && x.STATENAME == y.STATENAME ; } public int GetHashCode(USCensusPopulationData obj) { var hash = obj.ToString(); return hash.GetHashCode(); } } tag:odeo.com,2009-09-29,25203619 Tue, 29 Sep 2009 06:07:18 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25199910-Talking-with-Stefano-Mazzocchi-about-reconciling-web-naming-systems When Stefano Mazzocchi saw my posts on webscale identiers[1, 2] he pointed me to some recent work he and others have been doing at Metaweb. At ids.freebaseapps.com you can find sets of different web identifiers that refer to the same things. So, for example: Apple Inc. versus Apple Records Each of these views collects identifiers from different sources. For Apple Inc. they include: The NYTimes: topics.nytimes.com/top/news/business/companies/apple_computer_inc/ Wikipedia: wikipedia.org/wiki/Apple_Computer Open Library: openlibrary.org/a/OL2669993A/Inc._Apple_Computer On this week’s Innovators show Stefano joins me to discuss efforts underway at Metaweb to reconcile many different web naming systems and activate connections among them. Meanwhile my recent guest Kingsley Idehen is demonstrating a similar kind of name reconciliation at bbc.openlinksw.com. At this URL, for example, you can see canonical identifers for Michael Jackson from the BBC’s own namespace and others in... When Stefano Mazzocchi saw my posts on webscale identiers[1, 2] he pointed me to some recent work he and others have been doing at Metaweb. At ids.freebaseapps.com you can find sets of different web identifiers that refer to the same things. So, for example: Apple Inc. versus Apple Records Each of these views collects identifiers from different sources. For Apple Inc. they include: The NYTimes: topics.nytimes.com/top/news/business/companies/apple_computer_inc/ Wikipedia: wikipedia.org/wiki/Apple_Computer Open Library: openlibrary.org/a/OL2669993A/Inc._Apple_Computer On this week’s Innovators show Stefano joins me to discuss efforts underway at Metaweb to reconcile many different web naming systems and activate connections among them. Meanwhile my recent guest Kingsley Idehen is demonstrating a similar kind of name reconciliation at bbc.openlinksw.com. At this URL, for example, you can see canonical identifers for Michael Jackson from the BBC’s own namespace and others including DBpedia and OpenCyc. I’m not quite sure what to make of all this. But my spidey sense is telling me to pay attention, so I am. Related: Semantic web mashups for the rest of us A conversation with Stefano Mazzocchi about Cocoon and SIMILE Motivating people to write the semantic web: A conversation with David Huynh about Parallax Talking with Kingsley Idehen about mastering your own search index When Stefano Mazzocchi saw my posts on webscale identiers[1, 2] he pointed me to some recent work he and others have been doing at Metaweb. At ids.freebaseapps.com you can find sets of different web identifiers that refer to the same things. So, for example: Apple Inc. versus Apple Records Each of these views collects identifiers from different sources. For Apple Inc. they include: The NYTimes: topics.nytimes.com/top/news/business/companies/apple_computer_inc/ Wikipedia: wikipedia.org/wiki/Apple_Computer Open Library: openlibrary.org/a/OL2669993A/Inc._Apple_Computer On this week’s Innovators show Stefano joins me to discuss efforts underway at Metaweb to reconcile many different web naming systems and activate connections among them. Meanwhile my recent guest Kingsley Idehen is demonstrating a similar kind of name reconciliation at bbc.openlinksw.com. At this URL, for example, you can see canonical identifers for Michael Jackson from the BBC’s own namespace and others including DBpedia and OpenCyc. I’m not quite sure what to make of all this. But my spidey sense is telling me to pay attention, so I am. Related: Semantic web mashups for the rest of us A conversation with Stefano Mazzocchi about Cocoon and SIMILE Motivating people to write the semantic web: A conversation with David Huynh about Parallax Talking with Kingsley Idehen about mastering your own search index tag:odeo.com,2009-09-28,25199910 Mon, 28 Sep 2009 08:01:55 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25140838-Speaking-and-writing-webscale-identifiers I’ve really enjoyed the conversation about webscale identifiers. Naming web resources is such a crucial discipline, and yet one we’re all still making up as we go along. I ended the earlier post by suggesting that when we invent namespaces we should, where feasible, prefer names that make sense to people. In comments, a number of folks who have wrestled with the problem of ambiguity pointed out all sorts of reasons why that often just isn’t feasible. Gavin Bell likes Amazon’s hybrid approach: The model that Amazon have since moved to with a unique URL identifier and an ignored pretty human readable section is a good compromise. Michael Smethurst agreed with me that the BBC’s opaque IDs — for example, b006qpgr for The Archers — could be promoted as a tag vocabulary that people would be encouraged to use: Shownar is a prototype by Schulze and Webb that aims to track “buzz” around bbc programmes. For now it’s based on inbound links from b... I’ve really enjoyed the conversation about webscale identifiers. Naming web resources is such a crucial discipline, and yet one we’re all still making up as we go along. I ended the earlier post by suggesting that when we invent namespaces we should, where feasible, prefer names that make sense to people. In comments, a number of folks who have wrestled with the problem of ambiguity pointed out all sorts of reasons why that often just isn’t feasible. Gavin Bell likes Amazon’s hybrid approach: The model that Amazon have since moved to with a unique URL identifier and an ignored pretty human readable section is a good compromise. Michael Smethurst agreed with me that the BBC’s opaque IDs — for example, b006qpgr for The Archers — could be promoted as a tag vocabulary that people would be encouraged to use: Shownar is a prototype by Schulze and Webb that aims to track “buzz” around bbc programmes. For now it’s based on inbound links from blogs/twitter/etc but it could be expanded to use machine tags!?! On Shownar, I find that this episode of Miss Marple was discussed in this blog entry: BBC Radio have just started an Agatha Christie season and a whole host of programmes about the Queen of Crime are available to UK listeners on the iPlayer. They include dramatizations of works starring super sleuths from Miss Marple to the Mysterious Mr Quin, as well as revealing documentaries. The entry uses URLs that embed these BBC ids: b00mk71d, b007jvht. How did the author find them? Clearly, in this case, by way of the search URL which is also cited in the entry: http://www.bbc.co.uk/iplayer/search/?q= agatha christie The search term agatha christie is wildly ambiguous, of course. Shownar would never have included this item had it not cited specific BBC shows by way of their opaque IDs. Nor would the author have cited them if that had required typing b00mk71d or b007jvht. It only works thanks to copy/paste, but it works quite nicely, and it shows why site-specific search still matters in an era of uber search engines. This example got me thinking about the character strings that we can and do type, easily and naturally, versus those we can’t and won’t. For example: queries (what we can and do type) results (what we can’t and don’t type) http://www.librarything.com/catalog/jonudell&deepsearch= practical internet groupware http://www.librarything.com/work/ 16804 http://www.librarything.com/work/16804/book/ 28447984 http://www.google.com/search?q= practical internet groupware http://oreilly.com/catalog/ 9781565925373 http://oreilly.com/catalog/ pracintgr http://www.bing.com/results.aspx?q= practical internet groupware http://www.amazon.com/Practical-Internet-Groupware-Jon-Udell/dp/ 156592537 http://my.safaribooksonline.com/ 1565925378 http://www.worldcat.org/search?q= practical internet groupware http://www.worldcat.org/oclc/ 43188074 http://www.amazon.com/s?index=blended&field-keywords= practical internet groupware http://www.amazon.com/Practical-Internet-Groupware-Jon-Udell/dp/ 1565925378   Looking at the consistency on the left column, and the variation on the right, I’ve got to conclude that: Practical Internet Groupware is the de facto webscale identifier for my book. 16804, 28447984, 9781565925373, pracintgr, 156592537, 1565925378, and 43188074 will never converge. I’ve long imagined a class of equivalence services that would help us bridge the gap between vocabularies we can speak and write and those we’ll never speak and need help to write. Both are sets of webscale identifiers that we’ll need to use in complementary ways. That’ll require a mix of social conventions and technical services. I’ve really enjoyed the conversation about webscale identifiers. Naming web resources is such a crucial discipline, and yet one we’re all still making up as we go along. I ended the earlier post by suggesting that when we invent namespaces we should, where feasible, prefer names that make sense to people. In comments, a number of folks who have wrestled with the problem of ambiguity pointed out all sorts of reasons why that often just isn’t feasible. Gavin Bell likes Amazon’s hybrid approach: The model that Amazon have since moved to with a unique URL identifier and an ignored pretty human readable section is a good compromise. Michael Smethurst agreed with me that the BBC’s opaque IDs — for example, b006qpgr for The Archers — could be promoted as a tag vocabulary that people would be encouraged to use: Shownar is a prototype by Schulze and Webb that aims to track “buzz” around bbc programmes. For now it’s based on inbound links from blogs/twitter/etc but it could be expanded to use machine tags!?! On Shownar, I find that this episode of Miss Marple was discussed in this blog entry: BBC Radio have just started an Agatha Christie season and a whole host of programmes about the Queen of Crime are available to UK listeners on the iPlayer. They include dramatizations of works starring super sleuths from Miss Marple to the Mysterious Mr Quin, as well as revealing documentaries. The entry uses URLs that embed these BBC ids: b00mk71d, b007jvht. How did the author find them? Clearly, in this case, by way of the search URL which is also cited in the entry: http://www.bbc.co.uk/iplayer/search/?q= agatha christie The search term agatha christie is wildly ambiguous, of course. Shownar would never have included this item had it not cited specific BBC shows by way of their opaque IDs. Nor would the author have cited them if that had required typing b00mk71d or b007jvht. It only works thanks to copy/paste, but it works quite nicely, and it shows why site-specific search still matters in an era of uber search engines. This example got me thinking about the character strings that we can and do type, easily and naturally, versus those we can’t and won’t. For example: queries (what we can and do type) results (what we can’t and don’t type) http://www.librarything.com/catalog/jonudell&deepsearch= practical internet groupware http://www.librarything.com/work/ 16804 http://www.librarything.com/work/16804/book/ 28447984 http://www.google.com/search?q= practical internet groupware http://oreilly.com/catalog/ 9781565925373 http://oreilly.com/catalog/ pracintgr http://www.bing.com/results.aspx?q= practical internet groupware http://www.amazon.com/Practical-Internet-Groupware-Jon-Udell/dp/ 156592537 http://my.safaribooksonline.com/ 1565925378 http://www.worldcat.org/search?q= practical internet groupware http://www.worldcat.org/oclc/ 43188074 http://www.amazon.com/s?index=blended&field-keywords= practical internet groupware http://www.amazon.com/Practical-Internet-Groupware-Jon-Udell/dp/ 1565925378   Looking at the consistency on the left column, and the variation on the right, I’ve got to conclude that: Practical Internet Groupware is the de facto webscale identifier for my book. 16804, 28447984, 9781565925373, pracintgr, 156592537, 1565925378, and 43188074 will never converge. I’ve long imagined a class of equivalence services that would help us bridge the gap between vocabularies we can speak and write and those we’ll never speak and need help to write. Both are sets of webscale identifiers that we’ll need to use in complementary ways. That’ll require a mix of social conventions and technical services. tag:odeo.com,2009-09-17,25140838 Thu, 17 Sep 2009 09:54:02 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25135131-Familiar-idioms-in-Perl-Python-JavaScript-and-C When I started working on the elmcity project, I planned to use my language of choice in recent years: Python. But early on, IronPython wasn’t fully supported on Azure, so I switched to C#. Later, when IronPython became fully supported, there was really no point in switching my core roles (worker and web) to it, so I’ve proceeded in a hybrid mode. The core roles are written in C#, and a variety of auxiliary pieces are written in IronPython. Meanwhile, I’ve been creating other auxiliary pieces in JavaScript, as will happen with any web project. The other day, at the request of a calendar curator, I used JavaScript to prototype a tag summarizer. This was so useful that I decided to make it a new feature of the service. The C# version was so strikingly similar to the JavaScript version that I just had to set them side by side for comparison: JavaScript C# var tagdict = new Object(); for ( i = 0; i < obj.length; i++ ) { var evt = obj[i]; if ( evt["categories&qu... When I started working on the elmcity project, I planned to use my language of choice in recent years: Python. But early on, IronPython wasn’t fully supported on Azure, so I switched to C#. Later, when IronPython became fully supported, there was really no point in switching my core roles (worker and web) to it, so I’ve proceeded in a hybrid mode. The core roles are written in C#, and a variety of auxiliary pieces are written in IronPython. Meanwhile, I’ve been creating other auxiliary pieces in JavaScript, as will happen with any web project. The other day, at the request of a calendar curator, I used JavaScript to prototype a tag summarizer. This was so useful that I decided to make it a new feature of the service. The C# version was so strikingly similar to the JavaScript version that I just had to set them side by side for comparison: JavaScript C# var tagdict = new Object(); for ( i = 0; i < obj.length; i++ ) { var evt = obj[i]; if ( evt["categories"] != undefined) { var tags = evt["categories"].split(','); for (j = 0; j < tags.length; j++ ) { var tag = tags[j]; if ( tagdict[tag] != undefined ) tagdict[tag]++; else tagdict[tag] = 1; } } } var tagdict = new Dictionary(); foreach (var evt in es.events) { if (evt.categories != null) { var tags = evt.categories.Split(','); foreach (var tag in tags) { if (tagdict.ContainsKey(tag)) tagdict[tag]++; else tagdict[tag] = 1; } } } var sorted_keys = []; for ( var tag in tagdict ) sorted_keys.push(tag); sorted_keys.sort(function(a,b) { return tagdict[b] - tagdict[a] }); var sorted_keys = new List(); foreach (var tag in tagdict.Keys) sorted_keys.Add(tag); sorted_keys.Sort( (a, b) => tagdict[b].CompareTo(tagdict[a])); The idioms involved here include: Splitting a string on a delimiter to produce a list Using a dictionary to build a concordance of strings and occurrence counts Sorting an array of keys by their associated occurrence counts I first used these idioms in Perl. Later they became Python staples. Now here they are again, in both JavaScript and C#. When I started working on the elmcity project, I planned to use my language of choice in recent years: Python. But early on, IronPython wasn’t fully supported on Azure, so I switched to C#. Later, when IronPython became fully supported, there was really no point in switching my core roles (worker and web) to it, so I’ve proceeded in a hybrid mode. The core roles are written in C#, and a variety of auxiliary pieces are written in IronPython. Meanwhile, I’ve been creating other auxiliary pieces in JavaScript, as will happen with any web project. The other day, at the request of a calendar curator, I used JavaScript to prototype a tag summarizer. This was so useful that I decided to make it a new feature of the service. The C# version was so strikingly similar to the JavaScript version that I just had to set them side by side for comparison: JavaScript C# var tagdict = new Object(); for ( i = 0; i < obj.length; i++ ) { var evt = obj[i]; if ( evt["categories"] != undefined) { var tags = evt["categories"].split(','); for (j = 0; j < tags.length; j++ ) { var tag = tags[j]; if ( tagdict[tag] != undefined ) tagdict[tag]++; else tagdict[tag] = 1; } } } var tagdict = new Dictionary(); foreach (var evt in es.events) { if (evt.categories != null) { var tags = evt.categories.Split(','); foreach (var tag in tags) { if (tagdict.ContainsKey(tag)) tagdict[tag]++; else tagdict[tag] = 1; } } } var sorted_keys = []; for ( var tag in tagdict ) sorted_keys.push(tag); sorted_keys.sort(function(a,b) { return tagdict[b] - tagdict[a] }); var sorted_keys = new List(); foreach (var tag in tagdict.Keys) sorted_keys.Add(tag); sorted_keys.Sort( (a, b) => tagdict[b].CompareTo(tagdict[a])); The idioms involved here include: Splitting a string on a delimiter to produce a list Using a dictionary to build a concordance of strings and occurrence counts Sorting an array of keys by their associated occurrence counts I first used these idioms in Perl. Later they became Python staples. Now here they are again, in both JavaScript and C#. tag:odeo.com,2009-09-16,25135131 Wed, 16 Sep 2009 10:01:21 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25130084-Talking-with-Hugh-McGuire-about-BookOven On this week’s Innovators show I reconnect with Hugh McGuire. He’s the 104th guest in the current incarnation of the show, and was also the fourth. With Hugh it’s always about books and collaboration. Our first conversation explored one of my favorite projects, LibriVox, which brings people together to make free downloadable audiobooks. This time around we talked about his new project, BookOven, which aims to help authors, editors, and readers work together to create new books. Writing a book was the hardest thing I’ve ever done. The loneliness was what got to me. I finished around the time the blogosophere was starting to emerge, and the collegial joy I found here made me think I’d never want to repeat that solitary experience. Nowadays I wouldn’t have to. Authors commonly write books out in the open on blogs. BookOven aims to push that strategy further by providing a suite of online tools purpose-built for discussing, editing, and proofing long ... On this week’s Innovators show I reconnect with Hugh McGuire. He’s the 104th guest in the current incarnation of the show, and was also the fourth. With Hugh it’s always about books and collaboration. Our first conversation explored one of my favorite projects, LibriVox, which brings people together to make free downloadable audiobooks. This time around we talked about his new project, BookOven, which aims to help authors, editors, and readers work together to create new books. Writing a book was the hardest thing I’ve ever done. The loneliness was what got to me. I finished around the time the blogosophere was starting to emerge, and the collegial joy I found here made me think I’d never want to repeat that solitary experience. Nowadays I wouldn’t have to. Authors commonly write books out in the open on blogs. BookOven aims to push that strategy further by providing a suite of online tools purpose-built for discussing, editing, and proofing long texts. Given the rise of the 140-character blurb, this emphasis on the long form is counter-cyclical. But for me, at least, the pendulum is swinging back. Lately I’m snacking less on Twitter and enjoying full meals served up by the blogosphere, online magazines, library books. It’s been nourishing. But I’m also noticing that much of this work — in the commercial as well as the amateur realm — could benefit from better organization, editing, and proofing. The collaborative restructuring of all kinds of professional work has only just begun. Hugh McGuire and I share the belief that our new ability to harness what Yochai Benkler calls the loose affiliation of ad-hoc teams will yield better results in many areas. Book-length writing is the domain that Hugh has staked out. How can the new modes of collaboration enhance this ancient practice? We’ll see. On this week’s Innovators show I reconnect with Hugh McGuire. He’s the 104th guest in the current incarnation of the show, and was also the fourth. With Hugh it’s always about books and collaboration. Our first conversation explored one of my favorite projects, LibriVox, which brings people together to make free downloadable audiobooks. This time around we talked about his new project, BookOven, which aims to help authors, editors, and readers work together to create new books. Writing a book was the hardest thing I’ve ever done. The loneliness was what got to me. I finished around the time the blogosophere was starting to emerge, and the collegial joy I found here made me think I’d never want to repeat that solitary experience. Nowadays I wouldn’t have to. Authors commonly write books out in the open on blogs. BookOven aims to push that strategy further by providing a suite of online tools purpose-built for discussing, editing, and proofing long texts. Given the rise of the 140-character blurb, this emphasis on the long form is counter-cyclical. But for me, at least, the pendulum is swinging back. Lately I’m snacking less on Twitter and enjoying full meals served up by the blogosphere, online magazines, library books. It’s been nourishing. But I’m also noticing that much of this work — in the commercial as well as the amateur realm — could benefit from better organization, editing, and proofing. The collaborative restructuring of all kinds of professional work has only just begun. Hugh McGuire and I share the belief that our new ability to harness what Yochai Benkler calls the loose affiliation of ad-hoc teams will yield better results in many areas. Book-length writing is the domain that Hugh has staked out. How can the new modes of collaboration enhance this ancient practice? We’ll see. tag:odeo.com,2009-09-15,25130084 Tue, 15 Sep 2009 10:50:03 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25124369-Ask-and-ye-may-receive-don%E2%80%99t-ask-and-ye-surely-will-not This fall a small team of University of Toronto and Michigan State undergrads will be working on parts of the elmcity project by way of Undergraduate Capstone Open Source Projects (UCOSP), organized by Greg Wilson. In our first online meeting, the students decided they’d like to tackle the problem that FuseCal was solving: extraction of well-structured calendar information from weakly-structured web pages. From a computer science perspective, there’s a fairly obvious path. Start with specific examples that can be scraped, then work toward a more general solution. So the first two examples are going to be MySpace and LibraryThing. The recipes[1, 2] I’d concocted for FuseCal-written iCalendar feeds were especially valuable because they could be used by almost any curator for almost any location. But as I mentioned to the students, there’s another way to approach these two cases. And I was reminded of it again when Michael Foord pointed to this fascinating post ... This fall a small team of University of Toronto and Michigan State undergrads will be working on parts of the elmcity project by way of Undergraduate Capstone Open Source Projects (UCOSP), organized by Greg Wilson. In our first online meeting, the students decided they’d like to tackle the problem that FuseCal was solving: extraction of well-structured calendar information from weakly-structured web pages. From a computer science perspective, there’s a fairly obvious path. Start with specific examples that can be scraped, then work toward a more general solution. So the first two examples are going to be MySpace and LibraryThing. The recipes[1, 2] I’d concocted for FuseCal-written iCalendar feeds were especially valuable because they could be used by almost any curator for almost any location. But as I mentioned to the students, there’s another way to approach these two cases. And I was reminded of it again when Michael Foord pointed to this fascinating post prompted by the open source release of FriendFeed’s homegrown web server, Tornado. The author of the post, Glyph Lefkowitz, is the founder of Twisted, a Python-based network programming framework that includes the sort of asynchronous event-driven capabilities that FriendFeed recreated for Tornado. Glyph writes: If you’re about to undergo a re-write of a major project because it didn’t meet some requirements that you had, please tell the project that you are rewriting what you are doing. In the best case scenario, someone involved with that project will say, “Oh, you’ve misunderstood the documentation, actually it does do that”. In the worst case, you go ahead with your rewrite anyway, but there is some hope that you might be able to cooperate in the future, as the project gradually evolves to meet your requirements. Somewhere in the middle, you might be able to contribute a few small fixes rather than re-implementing the whole thing and maintaining it yourself. Whether FriendFeed could have improved the parts of Twisted that it found lacking, while leveraging its synergistic aspects, is a question only specialists close to both projects can answer. But Glyph is making a more general point. If you don’t communicate your intentions, such questions can never even be asked. Tying this back to the elmcity project, I mentioned to the students that the best scraper for MySpace and LibraryThing calendars is no scraper at all. If these services produced iCalendar feeds directly, there would be no need. That would be the ideal solution — a win for existing users of the services, and for the iCalendar ecosystem I’m trying to bootstrap. I’ve previously asked contacts at MySpace and LibraryThing about this. But now, since we’re intending to scrape those services for calendar info, it can’t hurt to announce that intention and hope one or both services will provide feeds directly and obviate the need. That way the students can focus on different problems — and there are plenty to choose from. So I’ll be sending the URL of this post to my contacts at those companies, and if any readers of this blog can help move things along, please do. We may end up with scrapers anyway. But maybe not. Maybe iCalendar feeds have already been provided, but aren’t documented. Maybe they were in the priority stack and this reminder will bump them up. It’s worth a shot. If the problem can be solved by communicating intentions rather than writing redundant code, that’s the ultimate hack. And its one that I hope more computer science students will learn to aspire to. This fall a small team of University of Toronto and Michigan State undergrads will be working on parts of the elmcity project by way of Undergraduate Capstone Open Source Projects (UCOSP), organized by Greg Wilson. In our first online meeting, the students decided they’d like to tackle the problem that FuseCal was solving: extraction of well-structured calendar information from weakly-structured web pages. From a computer science perspective, there’s a fairly obvious path. Start with specific examples that can be scraped, then work toward a more general solution. So the first two examples are going to be MySpace and LibraryThing. The recipes[1, 2] I’d concocted for FuseCal-written iCalendar feeds were especially valuable because they could be used by almost any curator for almost any location. But as I mentioned to the students, there’s another way to approach these two cases. And I was reminded of it again when Michael Foord pointed to this fascinating post prompted by the open source release of FriendFeed’s homegrown web server, Tornado. The author of the post, Glyph Lefkowitz, is the founder of Twisted, a Python-based network programming framework that includes the sort of asynchronous event-driven capabilities that FriendFeed recreated for Tornado. Glyph writes: If you’re about to undergo a re-write of a major project because it didn’t meet some requirements that you had, please tell the project that you are rewriting what you are doing. In the best case scenario, someone involved with that project will say, “Oh, you’ve misunderstood the documentation, actually it does do that”. In the worst case, you go ahead with your rewrite anyway, but there is some hope that you might be able to cooperate in the future, as the project gradually evolves to meet your requirements. Somewhere in the middle, you might be able to contribute a few small fixes rather than re-implementing the whole thing and maintaining it yourself. Whether FriendFeed could have improved the parts of Twisted that it found lacking, while leveraging its synergistic aspects, is a question only specialists close to both projects can answer. But Glyph is making a more general point. If you don’t communicate your intentions, such questions can never even be asked. Tying this back to the elmcity project, I mentioned to the students that the best scraper for MySpace and LibraryThing calendars is no scraper at all. If these services produced iCalendar feeds directly, there would be no need. That would be the ideal solution — a win for existing users of the services, and for the iCalendar ecosystem I’m trying to bootstrap. I’ve previously asked contacts at MySpace and LibraryThing about this. But now, since we’re intending to scrape those services for calendar info, it can’t hurt to announce that intention and hope one or both services will provide feeds directly and obviate the need. That way the students can focus on different problems — and there are plenty to choose from. So I’ll be sending the URL of this post to my contacts at those companies, and if any readers of this blog can help move things along, please do. We may end up with scrapers anyway. But maybe not. Maybe iCalendar feeds have already been provided, but aren’t documented. Maybe they were in the priority stack and this reminder will bump them up. It’s worth a shot. If the problem can be solved by communicating intentions rather than writing redundant code, that’s the ultimate hack. And its one that I hope more computer science students will learn to aspire to. tag:odeo.com,2009-09-14,25124369 Mon, 14 Sep 2009 09:15:26 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25098023-Talking-with-Kingsley-Idehen-about-mastering-your-own-search-index Kingsley Idehen’s vision of a web of linked data long predates the recognition I accorded him in 2003. He’s seen the big picture for a very long time, and has been driving toward it consistently. Over the years we’ve had conversations in which I’ve always wound up saying: “Yes, OK, but how will we get people to create this web of linked data that we want to navigate and query?” On this week’s Innovators show he responds with what I find to be a plausible scenario. Every business, and increasingly every person, presents some kind of home page to the world. On those pages you will find, implied but not clearly stated, one or both of the following kinds of assertions: 1. Things I offer. 2. Things I seek. A plumber, for example, may offer hydronic heating services, and may seek an assistant with certain qualifications. By encoding these kinds of assertions as subject-verb-object triples we could, in theory, build a semantic web that matches seek... Kingsley Idehen’s vision of a web of linked data long predates the recognition I accorded him in 2003. He’s seen the big picture for a very long time, and has been driving toward it consistently. Over the years we’ve had conversations in which I’ve always wound up saying: “Yes, OK, but how will we get people to create this web of linked data that we want to navigate and query?” On this week’s Innovators show he responds with what I find to be a plausible scenario. Every business, and increasingly every person, presents some kind of home page to the world. On those pages you will find, implied but not clearly stated, one or both of the following kinds of assertions: 1. Things I offer. 2. Things I seek. A plumber, for example, may offer hydronic heating services, and may seek an assistant with certain qualifications. By encoding these kinds of assertions as subject-verb-object triples we could, in theory, build a semantic web that matches seekers and finders more efficiently than the current searchable web can. But that first step was always doozy. Writing the assertions required an XML syntax which has never become a web mainstay. There are other ways to write them, however. Using an approach called RDFa, you can embed them directly into human-readable web pages. This isn’t a new idea. A decade ago, in my book Practical Internet Groupware, I showed how CSS class attributes could do double duty within a web page, governing style while also conveying meaning. In 2003 I was still experimenting with the idea, which I then called microcontent. Nowadays the term is microformats. Although we’ve heard plenty about this idea over the years, it has yet to bear fruit. I don’t know that it will, but the scenario Kingsley Idehen outlines strikes me as plausible because, as Dries Buytaert evocatively says, structured data is the new search engine optimization. Formerly of concern only to publishers, the rationale for search engine optimization is now becoming evident to everyone who writes an About page for their businesses or — what often comes to the same thing — for themselves. The formula for an About page is well known: name, address, services offered, hours of operation, etc. Everyone writes this stuff once for the About page, and then again in countless variations for inclusion in various directories. Kingsley and I both hope that the time is now ripe for a web-friendly way to write this data into About pages once, for common use by human visitors, search crawlers, and syndicated directories. His proposal relies on RDFa to encode factual assertions, and on an e-commerce ontology called GoodRelations which, as its creator Martin Hepp says, provides the vocabulary to say things like: a particular Web site describes an offer to sell cellphones of a certain make and model at a certain price, a pianohouse offers maintenance for pianos that weigh less than 150 kg, a car rental company leases out cars of a certain make and model from a particular set of branches across the country. The GoodRelations wiki shows cookbook examples for Yahoo and Google. You’d have to be fairly technical to adapt these using cut-and-paste, but there’s also a form that, although currently still wired to emit the older RDF/XML kinds of assertions, will soon also emit RDFa that can be woven into existing About pages. To navigate and query a web of linked data you need, obviously, mechanisms by which to do the navigation and the querying. That’s never been the problem. Technologists love to figure such things out. But we’ve spectacularly failed to help people create that web of linked data in the first place. I don’t know if the approach Kingsley Idehen sketches in this week’s podcast will succeed. But it feels right, and I love his tagline: “Be the master of your own index.” Kingsley Idehen’s vision of a web of linked data long predates the recognition I accorded him in 2003. He’s seen the big picture for a very long time, and has been driving toward it consistently. Over the years we’ve had conversations in which I’ve always wound up saying: “Yes, OK, but how will we get people to create this web of linked data that we want to navigate and query?” On this week’s Innovators show he responds with what I find to be a plausible scenario. Every business, and increasingly every person, presents some kind of home page to the world. On those pages you will find, implied but not clearly stated, one or both of the following kinds of assertions: 1. Things I offer. 2. Things I seek. A plumber, for example, may offer hydronic heating services, and may seek an assistant with certain qualifications. By encoding these kinds of assertions as subject-verb-object triples we could, in theory, build a semantic web that matches seekers and finders more efficiently than the current searchable web can. But that first step was always doozy. Writing the assertions required an XML syntax which has never become a web mainstay. There are other ways to write them, however. Using an approach called RDFa, you can embed them directly into human-readable web pages. This isn’t a new idea. A decade ago, in my book Practical Internet Groupware, I showed how CSS class attributes could do double duty within a web page, governing style while also conveying meaning. In 2003 I was still experimenting with the idea, which I then called microcontent. Nowadays the term is microformats. Although we’ve heard plenty about this idea over the years, it has yet to bear fruit. I don’t know that it will, but the scenario Kingsley Idehen outlines strikes me as plausible because, as Dries Buytaert evocatively says, structured data is the new search engine optimization. Formerly of concern only to publishers, the rationale for search engine optimization is now becoming evident to everyone who writes an About page for their businesses or — what often comes to the same thing — for themselves. The formula for an About page is well known: name, address, services offered, hours of operation, etc. Everyone writes this stuff once for the About page, and then again in countless variations for inclusion in various directories. Kingsley and I both hope that the time is now ripe for a web-friendly way to write this data into About pages once, for common use by human visitors, search crawlers, and syndicated directories. His proposal relies on RDFa to encode factual assertions, and on an e-commerce ontology called GoodRelations which, as its creator Martin Hepp says, provides the vocabulary to say things like: a particular Web site describes an offer to sell cellphones of a certain make and model at a certain price, a pianohouse offers maintenance for pianos that weigh less than 150 kg, a car rental company leases out cars of a certain make and model from a particular set of branches across the country. The GoodRelations wiki shows cookbook examples for Yahoo and Google. You’d have to be fairly technical to adapt these using cut-and-paste, but there’s also a form that, although currently still wired to emit the older RDF/XML kinds of assertions, will soon also emit RDFa that can be woven into existing About pages. To navigate and query a web of linked data you need, obviously, mechanisms by which to do the navigation and the querying. That’s never been the problem. Technologists love to figure such things out. But we’ve spectacularly failed to help people create that web of linked data in the first place. I don’t know if the approach Kingsley Idehen sketches in this week’s podcast will succeed. But it feels right, and I love his tagline: “Be the master of your own index.” tag:odeo.com,2009-09-09,25098023 Wed, 09 Sep 2009 07:21:49 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25050648-The-joy-of-webscale-identifiers My guest for this week’s Innovators show, Ian Forrester, heads up the BBC’s Backstage project. Launched in 2005, Backstage lives at a cultural crossroads where legacy systems and methods intersect with their next-generation counterparts. The tagline for the feeds and APIs provided under the Backstage umbrella is “use our stuff to build your stuff.” Admittedly that sounded a lot more exciting prior to 2006, when the BBC ended its trial of the Creative Archive service that was expected to “open the floodgates” to a “treasure trove” of cultural riches. Ian Forrester says those expectations were ratcheted back for two reasons. First, much of that treasure trove remains undigitized. Second, rights clearance proved to be an intractable problem. So the “our stuff” that’s available to build “your stuff” turns out to be mostly metadata: news headlines, program titles and schedules. What’s more, that metadata ... My guest for this week’s Innovators show, Ian Forrester, heads up the BBC’s Backstage project. Launched in 2005, Backstage lives at a cultural crossroads where legacy systems and methods intersect with their next-generation counterparts. The tagline for the feeds and APIs provided under the Backstage umbrella is “use our stuff to build your stuff.” Admittedly that sounded a lot more exciting prior to 2006, when the BBC ended its trial of the Creative Archive service that was expected to “open the floodgates” to a “treasure trove” of cultural riches. Ian Forrester says those expectations were ratcheted back for two reasons. First, much of that treasure trove remains undigitized. Second, rights clearance proved to be an intractable problem. So the “our stuff” that’s available to build “your stuff” turns out to be mostly metadata: news headlines, program titles and schedules. What’s more, that metadata comes from a plethora of BBC content management systems. What can you make out of these ingredients? Here’s an evocative example: http://www.bbc.co.uk/nature/species/African_Bush_Elephant. The BBC’s Tom Scott explains: Over the last few months we’ve been plundering the NHU’s [Natural History Unit's] archive to find the best bits — segmenting the TV programmes, tagging them (with DBpedia terms) and then aggregating them around URIs for the key concepts within the natural history domain; so that you can discover those programme segments via both the originating programme and via concepts within the natural history domain — species, habitats, adaptations and the like. This is just the sort of remixing that Backstage ought to enable anyone, inside or outside the BBC, to achieve. Since I’m a US resident, and don’t pay the UK’s television license fee, I can’t watch the videos on that page. There’s nothing that the Backstage team can do about that. But they can take a radically open and inclusive approach to the management of the metadata that supports this remixing, and that’s just what they’re doing. In our conversation, Ian Forrester describes how the taxonomy that governs the Backstage feeds and APIs is shared with that of Wikipedia and its structured derivative, DBpedia. Tom Scott elaborates: You might have noticed that the slugs for our URIs (the last bit of the URL) are the same as those used by Wikipedia and DBpedia that’s because I believe in the simple joy of webscale identifiers, you will also see that much like the BBC’s music site we are transcluding the introductory text from Wikipedia to provide background information for most things. This also means that we are creating and editing Wikipedia articles where they need improving (of course you are also more than welcome to improve upon the articles). As someone who both practices and preaches collaborative curation, I’m delighted to see the BBC taking this approach. And I love the phrase webscale identifier. Here’s how Michael Smethurst defines it, in the post pointed to by Tom Scott: I agree with the four Linked Data rules but I’d like to try to add a fifth: if possible don’t reinvent other people’s web identifiers. By web identifiers I mean those fragments of URLs that uniquely identify a resource within a domain. So in the case of the MusicBrainz entry for The Fall (http://musicbrainz.org/artist/d5da1841-9bc8-4813-9f89-11098090148e.html) that’ll be d5da1841-9bc8-4813-9f89-11098090148e. The last time we updated the /music site we made this mistake (kind of unavoidable at the time). Even though we linked our data to MusicBrainz we minted new identifiers for artists. So The Fall became http://www.bbc.co.uk/music/artist/jb9x/ where jb9x was the identifier. But jb9x doesn’t exist anywhere outside of /music. We’ll (hopefully) never make that mistake again. Beautifully said. Enormous synergies have gone unrealized because web publishers have chosen to mint new namespaces rather than add value to existing ones. What I realized when talking with Ian, though, is that there is one namespace for which the BBC is the appropriate mint, namely its own. Here, for example, are some of the family of URLs for a radio drama called The Archers: homepage: http://www.bbc.co.uk/programmes/b006qpgr/ upcoming shows: http://www.bbc.co.uk/programmes/b006qpgr/episodes/upcoming.xml In this example b006qpgr is, at least potentially, a webscale identifier. It’s a unique tag for the show that, if used on blogs, on Twitter, and elsewhere, would make it easy to assemble all kinds of online activity related to the show. But in fact only web developers using Backstage feeds and APIs will ever discover, or use, b006qpgr. In colloquial discourse people use The Archers. If the BBC wants people to collaborate with its namespace in the same way that it collaborates with Wikipedia’s, this would be more inviting: http://www.bbc.co.uk/programmes/The_Archers/ http://www.bbc.co.uk/programmes/The_Archers/episodes/upcoming.xml It should go without saying, but right after the first rule for linked data, “Use URIs as names for things,” I would add “Where possible, choose names that make sense to people.” My guest for this week’s Innovators show, Ian Forrester, heads up the BBC’s Backstage project. Launched in 2005, Backstage lives at a cultural crossroads where legacy systems and methods intersect with their next-generation counterparts. The tagline for the feeds and APIs provided under the Backstage umbrella is “use our stuff to build your stuff.” Admittedly that sounded a lot more exciting prior to 2006, when the BBC ended its trial of the Creative Archive service that was expected to “open the floodgates” to a “treasure trove” of cultural riches. Ian Forrester says those expectations were ratcheted back for two reasons. First, much of that treasure trove remains undigitized. Second, rights clearance proved to be an intractable problem. So the “our stuff” that’s available to build “your stuff” turns out to be mostly metadata: news headlines, program titles and schedules. What’s more, that metadata comes from a plethora of BBC content management systems. What can you make out of these ingredients? Here’s an evocative example: http://www.bbc.co.uk/nature/species/African_Bush_Elephant. The BBC’s Tom Scott explains: Over the last few months we’ve been plundering the NHU’s [Natural History Unit's] archive to find the best bits — segmenting the TV programmes, tagging them (with DBpedia terms) and then aggregating them around URIs for the key concepts within the natural history domain; so that you can discover those programme segments via both the originating programme and via concepts within the natural history domain — species, habitats, adaptations and the like. This is just the sort of remixing that Backstage ought to enable anyone, inside or outside the BBC, to achieve. Since I’m a US resident, and don’t pay the UK’s television license fee, I can’t watch the videos on that page. There’s nothing that the Backstage team can do about that. But they can take a radically open and inclusive approach to the management of the metadata that supports this remixing, and that’s just what they’re doing. In our conversation, Ian Forrester describes how the taxonomy that governs the Backstage feeds and APIs is shared with that of Wikipedia and its structured derivative, DBpedia. Tom Scott elaborates: You might have noticed that the slugs for our URIs (the last bit of the URL) are the same as those used by Wikipedia and DBpedia that’s because I believe in the simple joy of webscale identifiers, you will also see that much like the BBC’s music site we are transcluding the introductory text from Wikipedia to provide background information for most things. This also means that we are creating and editing Wikipedia articles where they need improving (of course you are also more than welcome to improve upon the articles). As someone who both practices and preaches collaborative curation, I’m delighted to see the BBC taking this approach. And I love the phrase webscale identifier. Here’s how Michael Smethurst defines it, in the post pointed to by Tom Scott: I agree with the four Linked Data rules but I’d like to try to add a fifth: if possible don’t reinvent other people’s web identifiers. By web identifiers I mean those fragments of URLs that uniquely identify a resource within a domain. So in the case of the MusicBrainz entry for The Fall (http://musicbrainz.org/artist/d5da1841-9bc8-4813-9f89-11098090148e.html) that’ll be d5da1841-9bc8-4813-9f89-11098090148e. The last time we updated the /music site we made this mistake (kind of unavoidable at the time). Even though we linked our data to MusicBrainz we minted new identifiers for artists. So The Fall became http://www.bbc.co.uk/music/artist/jb9x/ where jb9x was the identifier. But jb9x doesn’t exist anywhere outside of /music. We’ll (hopefully) never make that mistake again. Beautifully said. Enormous synergies have gone unrealized because web publishers have chosen to mint new namespaces rather than add value to existing ones. What I realized when talking with Ian, though, is that there is one namespace for which the BBC is the appropriate mint, namely its own. Here, for example, are some of the family of URLs for a radio drama called The Archers: homepage: http://www.bbc.co.uk/programmes/b006qpgr/ upcoming shows: http://www.bbc.co.uk/programmes/b006qpgr/episodes/upcoming.xml In this example b006qpgr is, at least potentially, a webscale identifier. It’s a unique tag for the show that, if used on blogs, on Twitter, and elsewhere, would make it easy to assemble all kinds of online activity related to the show. But in fact only web developers using Backstage feeds and APIs will ever discover, or use, b006qpgr. In colloquial discourse people use The Archers. If the BBC wants people to collaborate with its namespace in the same way that it collaborates with Wikipedia’s, this would be more inviting: http://www.bbc.co.uk/programmes/The_Archers/ http://www.bbc.co.uk/programmes/The_Archers/episodes/upcoming.xml It should go without saying, but right after the first rule for linked data, “Use URIs as names for things,” I would add “Where possible, choose names that make sense to people.” tag:odeo.com,2009-08-31,25050648 Mon, 31 Aug 2009 09:55:18 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25032237-FriendFeed-for-project-collaboration For me, FriendFeed has been a new answer to an old question — namely, how to collaborate in a loosely-coupled way with people who are using, and helping to develop, an online service. The elmcity project’s FriendFeed room has been an incredibly simple and effective way to interleave curated calendar feeds, blog postings describing the evolving service that aggregates those feeds, and discussion among a growing number of curators. In his analysis of Where FriendFeed Went Wrong Dare Obasanjo describes the value of a handful of services (Facebook, Twitter, etc.) in terms that would make sense to non-geeks like his wife. Here’s the elevator pitch for FriendFeed: Republish all of the content from the different social networking media websites you use onto this site. Also one place to stay connected to what people are saying on multiple social media sites instead of friending them on multiple sites. As usual, I’m an outlying data point. I’m using FriendFeed a... For me, FriendFeed has been a new answer to an old question — namely, how to collaborate in a loosely-coupled way with people who are using, and helping to develop, an online service. The elmcity project’s FriendFeed room has been an incredibly simple and effective way to interleave curated calendar feeds, blog postings describing the evolving service that aggregates those feeds, and discussion among a growing number of curators. In his analysis of Where FriendFeed Went Wrong Dare Obasanjo describes the value of a handful of services (Facebook, Twitter, etc.) in terms that would make sense to non-geeks like his wife. Here’s the elevator pitch for FriendFeed: Republish all of the content from the different social networking media websites you use onto this site. Also one place to stay connected to what people are saying on multiple social media sites instead of friending them on multiple sites. As usual, I’m an outlying data point. I’m using FriendFeed as a lightweight, flexible aggregator of feeds from my blog and from Delicious, and as a discussion forum. These feeds report key events in the life of the project: I added a new feature to the aggregator, the curator for Sasktatoon found and added a new calendar. The discussion revolves around strategies for finding or creating calendar feeds, features that curators would like me to add to the service, and problems they’re having with the service. I doubt there’s a mainstream business model here. It’s valuable to me because I’ve created a project environment in which key events in the life of the project are already flowing through feeds that are available to be aggregated and discussed. Anyone could arrange things that way, but few people will. It’s hugely helpful to me, though. And while I don’t know for sure that FriendFeed’s acquisition by FaceBook will end my ability to use FriendFeed in this way, I do need to start thinking about how I’d replace the service. I don’t need a lot of what FriendFeed offers. Many of the services it can aggregate — Flickr, YouTube, SlideShare — aren’t relevant. And we don’t need realtime notification. So it really boils down to a lightweight feed aggregator married to a discussion forum. One feature that FriendFeed’s API doesn’t offer, by the way, but that I would find useful, is programmatic control of the aggregator’s registry. When a new curator shows up, I have to manually add the associated Delicious feed to the FriendFeed room. It’d be nice to automate that. Ideally FriendFeed will coast along in a way that lets me keep using it as I currently am. If not, it wouldn’t be too hard to recreate something that provides just the subset of FriendFeed’s services that I need. But ideally, of course, I’d repurpose an existing service rather than build a new one. If you’re using something that could work, let me know. For me, FriendFeed has been a new answer to an old question — namely, how to collaborate in a loosely-coupled way with people who are using, and helping to develop, an online service. The elmcity project’s FriendFeed room has been an incredibly simple and effective way to interleave curated calendar feeds, blog postings describing the evolving service that aggregates those feeds, and discussion among a growing number of curators. In his analysis of Where FriendFeed Went Wrong Dare Obasanjo describes the value of a handful of services (Facebook, Twitter, etc.) in terms that would make sense to non-geeks like his wife. Here’s the elevator pitch for FriendFeed: Republish all of the content from the different social networking media websites you use onto this site. Also one place to stay connected to what people are saying on multiple social media sites instead of friending them on multiple sites. As usual, I’m an outlying data point. I’m using FriendFeed as a lightweight, flexible aggregator of feeds from my blog and from Delicious, and as a discussion forum. These feeds report key events in the life of the project: I added a new feature to the aggregator, the curator for Sasktatoon found and added a new calendar. The discussion revolves around strategies for finding or creating calendar feeds, features that curators would like me to add to the service, and problems they’re having with the service. I doubt there’s a mainstream business model here. It’s valuable to me because I’ve created a project environment in which key events in the life of the project are already flowing through feeds that are available to be aggregated and discussed. Anyone could arrange things that way, but few people will. It’s hugely helpful to me, though. And while I don’t know for sure that FriendFeed’s acquisition by FaceBook will end my ability to use FriendFeed in this way, I do need to start thinking about how I’d replace the service. I don’t need a lot of what FriendFeed offers. Many of the services it can aggregate — Flickr, YouTube, SlideShare — aren’t relevant. And we don’t need realtime notification. So it really boils down to a lightweight feed aggregator married to a discussion forum. One feature that FriendFeed’s API doesn’t offer, by the way, but that I would find useful, is programmatic control of the aggregator’s registry. When a new curator shows up, I have to manually add the associated Delicious feed to the FriendFeed room. It’d be nice to automate that. Ideally FriendFeed will coast along in a way that lets me keep using it as I currently am. If not, it wouldn’t be too hard to recreate something that provides just the subset of FriendFeed’s services that I need. But ideally, of course, I’d repurpose an existing service rather than build a new one. If you’re using something that could work, let me know. tag:odeo.com,2009-08-27,25032237 Thu, 27 Aug 2009 09:30:14 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/25020361-Purple-Numbers-for-PDF-documents My contribution to Silona Bonewald week was an interview about her new project citability.org. Silona proposes two new features for government websites. First, change tracking. Second, permalinks for documents, sections, and paragraphs. Nobody will dispute the need for, or utility of, these features. The question is how to implement them across a sprawling landscape of content management systems and publishing procedures that still, in many cases, regard print as canonical and the web as an afterthought. In a follow-on discussion with Silona, on the citability wiki, I recalled a little-known and rarely-used feature of PDF documents. You can form URLs that point to specific pages. And with the right preparation, you can even form URLs that point to named destinations within pages. Those of us fluent in web-friendly document formats like HTML and XML will tend to recommend that these become canonical. But having recently observed what happened when the old-fashioned non-XML method of ... My contribution to Silona Bonewald week was an interview about her new project citability.org. Silona proposes two new features for government websites. First, change tracking. Second, permalinks for documents, sections, and paragraphs. Nobody will dispute the need for, or utility of, these features. The question is how to implement them across a sprawling landscape of content management systems and publishing procedures that still, in many cases, regard print as canonical and the web as an afterthought. In a follow-on discussion with Silona, on the citability wiki, I recalled a little-known and rarely-used feature of PDF documents. You can form URLs that point to specific pages. And with the right preparation, you can even form URLs that point to named destinations within pages. Those of us fluent in web-friendly document formats like HTML and XML will tend to recommend that these become canonical. But having recently observed what happened when the old-fashioned non-XML method of math typesetting was supported by WordPress.com, I have to ask: How much more mileage might we be getting out of the existing print-oriented systems? I am not an expert user of PDF authoring tools, nor an expert user of software libraries that enable programmatic manipulation of PDF files. But some of you are. What would it take, I wonder, to post-process the kinds of PDF files that governments typically produce, in order to add Purple Numbers? My contribution to Silona Bonewald week was an interview about her new project citability.org. Silona proposes two new features for government websites. First, change tracking. Second, permalinks for documents, sections, and paragraphs. Nobody will dispute the need for, or utility of, these features. The question is how to implement them across a sprawling landscape of content management systems and publishing procedures that still, in many cases, regard print as canonical and the web as an afterthought. In a follow-on discussion with Silona, on the citability wiki, I recalled a little-known and rarely-used feature of PDF documents. You can form URLs that point to specific pages. And with the right preparation, you can even form URLs that point to named destinations within pages. Those of us fluent in web-friendly document formats like HTML and XML will tend to recommend that these become canonical. But having recently observed what happened when the old-fashioned non-XML method of math typesetting was supported by WordPress.com, I have to ask: How much more mileage might we be getting out of the existing print-oriented systems? I am not an expert user of PDF authoring tools, nor an expert user of software libraries that enable programmatic manipulation of PDF files. But some of you are. What would it take, I wonder, to post-process the kinds of PDF files that governments typically produce, in order to add Purple Numbers? tag:odeo.com,2009-08-25,25020361 Tue, 25 Aug 2009 06:52:11 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24967156-elmcity-and-WordPress-MU-Questions-and-answers In the spirit of keystroke conservation, I’m relaying some elmcity-related questions and answers from email to here. Hopefully it will attract more questions and more answers. Dear Mr. Udell, I am looking for a flexible calendar aggregator that I can use to report upcoming events for our college’s “Learning Commons” WordPress MU website, a site that will hopefully help keep our students abreast of events and opportunities taking place on campus. 1) Our site will be maintained using WordPress MU, so ideally the display of the calendars, and/or event-lists will be handled by a WordPress plugin. The one I am favouring is http://wordpress.org/extend/plugins/wordpress-ics-importer/ . I have tried this plugin and it almost does what we want. Specifically, the plugin includes: – a single widget that can display the “event-list” for one calendar; – flexible options for displaying and aggregating calendars. This plugin almost does what I want, ... In the spirit of keystroke conservation, I’m relaying some elmcity-related questions and answers from email to here. Hopefully it will attract more questions and more answers. Dear Mr. Udell, I am looking for a flexible calendar aggregator that I can use to report upcoming events for our college’s “Learning Commons” WordPress MU website, a site that will hopefully help keep our students abreast of events and opportunities taking place on campus. 1) Our site will be maintained using WordPress MU, so ideally the display of the calendars, and/or event-lists will be handled by a WordPress plugin. The one I am favouring is http://wordpress.org/extend/plugins/wordpress-ics-importer/ . I have tried this plugin and it almost does what we want. Specifically, the plugin includes: – a single widget that can display the “event-list” for one calendar; – flexible options for displaying and aggregating calendars. This plugin almost does what I want, but not quite. a) The plugin is now limited to a single “events-list” widget. But with WordPress 2.8, it is possible to have many instances of a widget, so theoretically, I could display the “Diagnostic Tests” calendar in one instance , and the “Peer-tutoring” calendar in another widget instance. b) It would be nice to have an option to display only the current week for specific calendars. While in other cases, it makes sense to display the entire month. And although I haven’t thought about it, likely displaying just the current day would be useful. c) I would like flexibility over which calendars to aggregate, creating as many “topic” hubs as the current maintainer of the website might think useful for the students. 2) It would be nice to remove the calendar aggregation from the WordPress plugin, and handle that separately. Hopefully the calendars will change much less frequently than the website will be viewed. If I understand http://blog.jonudell.net/elmcity-project-faq/ properly, this might be possible using the elmcity-project. For example, I think we could use “topical hub aggregation” to create a “diagnostic test calendar” that aggregates the holiday calendar and the different departments “diagnostic test” calendars. What I don’t understand is what is the output of “elmcity”. Does it output a single merged calendar (ics) that could be displayed by the above plugin? Is that a possibility? Similarly, I believe I could create a different meta bookmark to aggregate our holiday calendar and our different peer-tutoring calendars (created by each department). Is this correct? We have lots of groups, faculty, departments and staff on campus, and each wants to publicize their upcoming events. Letting them input and maintain their own calendars really seems to make sense. (Thanks for the idea. It seems clear this is the way to go, but I don’t seem to have the pieces to construct the output properly, as yet.) I agree with your analysis that it would be better to have a separation of concerns between aggregation and display. So let’s do that, and start with aggregation. I would like flexibility over which calendars to aggregate, creating as many “topic” hubs as the current maintainer of the website might think useful for the students. I think the elmcity system can be helpful here. I’ve recently discovered that there are really two levels — what I’ve started to call curation and meta-curation. I believe I could create bookmarks to aggregate our holiday calendar and our different peer-tutoring calendars (created by each department). Is this correct? Right. It sounds like you’d want to curate a topic hub. It could be YourCollege, but if there may need to be other topic hubs you could choose a more specific name, like YourCollegeLearningCommons. That’d be your Delicious account name, and you’d be the “meta-curator” in this scenario. As meta-curator you’d bookmark, in that Delicious account: - Your holiday calendar - Multiple departments’ calendars Each of those would be managed by the responsible/authoritative person, using any software (Outlook, Google, Apple, Drupal, Notes, Live, etc.) that can publish an ICS feed. There’s another level of flexibility using tags. In the above scenario, as meta-curator you could tag your holiday feed as holiday, and your LearningCommons feeds as LearningCommons, and then filter them accordingly. What I don’t understand is what is the output of elmcity. Does it output a single merged calendar (ics) that could be displayed by the above plugin? Yes. The outputs currently are: A merged ICS file A default HTML view that can be included in an iframe (e.g.: http://elmcity.info/events includes http://elmcity.cloudapp.net/services/elmcity/html). If that solution works for you, the template and CSS for the view can be altered, and you can point to your versions from your Delicious metadata A Today view (e.g. http://elmcity.cloudapp.net/services/elmcity/today_as_html) that can be included (in an iframe, or programmatically) The same Today view using JavaScript: <script src=”http://elmcity.cloudapp.net/services/{a2cal|elmcity|etc}/jswidget”></script> XML and JSON data for programmatic use Now, for the display options. So far, we’ve got: Use the WordPress plugin to display merged ICS Display the entire calendar as included (maybe customized) HTML Display today’s events as included or script-sourced HTML I have also just recently added a new method that enables things like this: http://jonudell.net/test/upcoming-widget.html You can view the source to see how it’s done. The “API call” here is: http://elmcity.cloudapp.net/services/elmcity/json?jsonp=eventlist&recent=7&view=music Yours might be: http://elmcity.cloudapp.net/services/YourCollegeLearningCommons/json?jsonp=eventlist&recent=10 or &recent=20&view=holiday etc. This is brand new, as of yesterday. Actually I just realized I should use “upcoming” instead of “recent” so I’ll go and change that now :-) But you get the idea. The flexibility here is ultimately governed by: 1. The curator’s expressive and disciplined use of tags to create useful views 2. The kinds of queries I make available through the API. So far I’ve only been asked to do ‘next N events’ so that’s what I did yesterday. But my intention is to support every kind of query that’s feasible, and that people ask for. Things like a week’s worth, or a week’s worth in a category, are obvious next steps. In the spirit of keystroke conservation, I’m relaying some elmcity-related questions and answers from email to here. Hopefully it will attract more questions and more answers. Dear Mr. Udell, I am looking for a flexible calendar aggregator that I can use to report upcoming events for our college’s “Learning Commons” WordPress MU website, a site that will hopefully help keep our students abreast of events and opportunities taking place on campus. 1) Our site will be maintained using WordPress MU, so ideally the display of the calendars, and/or event-lists will be handled by a WordPress plugin. The one I am favouring is http://wordpress.org/extend/plugins/wordpress-ics-importer/ . I have tried this plugin and it almost does what we want. Specifically, the plugin includes: – a single widget that can display the “event-list” for one calendar; – flexible options for displaying and aggregating calendars. This plugin almost does what I want, but not quite. a) The plugin is now limited to a single “events-list” widget. But with WordPress 2.8, it is possible to have many instances of a widget, so theoretically, I could display the “Diagnostic Tests” calendar in one instance , and the “Peer-tutoring” calendar in another widget instance. b) It would be nice to have an option to display only the current week for specific calendars. While in other cases, it makes sense to display the entire month. And although I haven’t thought about it, likely displaying just the current day would be useful. c) I would like flexibility over which calendars to aggregate, creating as many “topic” hubs as the current maintainer of the website might think useful for the students. 2) It would be nice to remove the calendar aggregation from the WordPress plugin, and handle that separately. Hopefully the calendars will change much less frequently than the website will be viewed. If I understand http://blog.jonudell.net/elmcity-project-faq/ properly, this might be possible using the elmcity-project. For example, I think we could use “topical hub aggregation” to create a “diagnostic test calendar” that aggregates the holiday calendar and the different departments “diagnostic test” calendars. What I don’t understand is what is the output of “elmcity”. Does it output a single merged calendar (ics) that could be displayed by the above plugin? Is that a possibility? Similarly, I believe I could create a different meta bookmark to aggregate our holiday calendar and our different peer-tutoring calendars (created by each department). Is this correct? We have lots of groups, faculty, departments and staff on campus, and each wants to publicize their upcoming events. Letting them input and maintain their own calendars really seems to make sense. (Thanks for the idea. It seems clear this is the way to go, but I don’t seem to have the pieces to construct the output properly, as yet.) I agree with your analysis that it would be better to have a separation of concerns between aggregation and display. So let’s do that, and start with aggregation. I would like flexibility over which calendars to aggregate, creating as many “topic” hubs as the current maintainer of the website might think useful for the students. I think the elmcity system can be helpful here. I’ve recently discovered that there are really two levels — what I’ve started to call curation and meta-curation. I believe I could create bookmarks to aggregate our holiday calendar and our different peer-tutoring calendars (created by each department). Is this correct? Right. It sounds like you’d want to curate a topic hub. It could be YourCollege, but if there may need to be other topic hubs you could choose a more specific name, like YourCollegeLearningCommons. That’d be your Delicious account name, and you’d be the “meta-curator” in this scenario. As meta-curator you’d bookmark, in that Delicious account: - Your holiday calendar - Multiple departments’ calendars Each of those would be managed by the responsible/authoritative person, using any software (Outlook, Google, Apple, Drupal, Notes, Live, etc.) that can publish an ICS feed. There’s another level of flexibility using tags. In the above scenario, as meta-curator you could tag your holiday feed as holiday, and your LearningCommons feeds as LearningCommons, and then filter them accordingly. What I don’t understand is what is the output of elmcity. Does it output a single merged calendar (ics) that could be displayed by the above plugin? Yes. The outputs currently are: A merged ICS file A default HTML view that can be included in an iframe (e.g.: http://elmcity.info/events includes http://elmcity.cloudapp.net/services/elmcity/html). If that solution works for you, the template and CSS for the view can be altered, and you can point to your versions from your Delicious metadata A Today view (e.g. http://elmcity.cloudapp.net/services/elmcity/today_as_html) that can be included (in an iframe, or programmatically) The same Today view using JavaScript: <script src=”http://elmcity.cloudapp.net/services/{a2cal|elmcity|etc}/jswidget”></script> XML and JSON data for programmatic use Now, for the display options. So far, we’ve got: Use the WordPress plugin to display merged ICS Display the entire calendar as included (maybe customized) HTML Display today’s events as included or script-sourced HTML I have also just recently added a new method that enables things like this: http://jonudell.net/test/upcoming-widget.html You can view the source to see how it’s done. The “API call” here is: http://elmcity.cloudapp.net/services/elmcity/json?jsonp=eventlist&recent=7&view=music Yours might be: http://elmcity.cloudapp.net/services/YourCollegeLearningCommons/json?jsonp=eventlist&recent=10 or &recent=20&view=holiday etc. This is brand new, as of yesterday. Actually I just realized I should use “upcoming” instead of “recent” so I’ll go and change that now :-) But you get the idea. The flexibility here is ultimately governed by: 1. The curator’s expressive and disciplined use of tags to create useful views 2. The kinds of queries I make available through the API. So far I’ve only been asked to do ‘next N events’ so that’s what I did yesterday. But my intention is to support every kind of query that’s feasible, and that people ask for. Things like a week’s worth, or a week’s worth in a category, are obvious next steps. tag:odeo.com,2009-08-15,24967156 Sat, 15 Aug 2009 09:30:35 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24938744-Two-projects-for-civic-minded-student-programmers One of the key findings of the elmcity project, so far, is that there’s a lot of calendar information online, but very little in machine-readable form. Transforming this implicit data about public events into explicit data is an important challenge. I’ve been invited to define the problem, for students who may want to tackle it as a school project. Here are the two major aspects I’ve identified. A general scraper for calendar-like web pages There are zillions of calendar-like web pages, like this one for Harlow’s Pub in Peterborough, NH. These ideally ought to be maintained using calendar programs that publish machine-readable iCalendar feeds which are also transformed and styled to create human-readable web pages. But that doesn’t (yet) commonly happen. These web pages are, however, often amenable to scraping. And for a while, elmcity curators were making very effective use of FuseCal (1, 2, 3) to transform these kinds of pages into iCalendar feeds. Wh... One of the key findings of the elmcity project, so far, is that there’s a lot of calendar information online, but very little in machine-readable form. Transforming this implicit data about public events into explicit data is an important challenge. I’ve been invited to define the problem, for students who may want to tackle it as a school project. Here are the two major aspects I’ve identified. A general scraper for calendar-like web pages There are zillions of calendar-like web pages, like this one for Harlow’s Pub in Peterborough, NH. These ideally ought to be maintained using calendar programs that publish machine-readable iCalendar feeds which are also transformed and styled to create human-readable web pages. But that doesn’t (yet) commonly happen. These web pages are, however, often amenable to scraping. And for a while, elmcity curators were making very effective use of FuseCal (1, 2, 3) to transform these kinds of pages into iCalendar feeds. When that service shut down, I retained a list of the pages that elmcity curators were successfully transforming into iCalendar feeds using FuseCal. These are test cases for an HTML-to-iCalendar service. Anyone who’s handy with scraping libraries like Beautiful Soup can solve these individually. The challenge here is to create, by abstraction and generalization, an engine that can handle a significant swath of these cases. A hybrid system for finding implicit recurring events and making them explicit Lots of implicit calendar data online doesn’t even pretend to be calendar-like, and cannot be harvested using a scraper. Finding one-off events in this category is out of scope for my project. But finding recurring events seems promising. The singular effort required to publish one of these will pay ongoing dividends. It’s helpful that the language people use to describe these events — “every Tuesday”, “third Saturday of every month” — is distinctive. To being exploring this domain, I wrote a specialized search robot that looks for these patterns, in conjunction with names of places. Its output is available for all the cities and towns participating in the elmcity project. For example, this page is the output for Keene, NH. It includes more than 2000 links to web pages — or, quite often, PDF files — some fraction of which represent recurring events. In Finding and connecting social capital I showed a couple of cases where the pages found this way did, in fact, represent recurring events that could be added to an iCalendar feed. To a computer scientist this looks like a problem that you might solve using a natural language parser. And I think it is partly that, but only partly. Let’s look at another example: At first glance, this looks hopeful: First Monday of each month: Dads Group, 105 Castle Street, Keene NH But the real world is almost always messier than that. For starters, that image comes from the Monadnock Men’s Resource Center’s Fall 2004 newsletter. So before I add this to a calendar, I’ll want to confirm the information. The newsletter is hosted at the MMRC site. Investigation yields these observations: The most recent issue of the newsletter was Winter ‘06 The last-modified date of the MMRC home page is September 2008 As of that date, the Dads Group still seems to have been active, under a slightly different name: Parent Outreach Project, DadTime Program, 355-3082 There’s no email address, only a phone number. So I called the number, left a message, and will soon know the current status. What kind of software-based system can help us scale this gnarly process? There is an algorithmic solution, surely, but it will need to operate in a hybrid environment. The initial search-driven discovery of candidate events can be done by an automated parser tuned for this domain. But the verification of candidates will need to be done by human volunteers, assisted by software that helps them: Divide long lists of candidates into smaller batches Work in parallel on those batches Evaluate the age and provenance of candidates Verify or disqualify candidates based on discoverable evidence, if possible Otherwise, find appropriate email addresses (preferably) or phone numbers, and manage the back-and-forth communication required to verify or disqualify a candidate Refer event sponsors to a calendar publishing how-to, and invite them to create data feeds that can reliably syndicate Students endowed with the geek gene are likely to gravitate toward the first problem because it’s cleaner. But I hope I can also attract interest in the second problem. We really need people who can hack that kind of real-world messiness. One of the key findings of the elmcity project, so far, is that there’s a lot of calendar information online, but very little in machine-readable form. Transforming this implicit data about public events into explicit data is an important challenge. I’ve been invited to define the problem, for students who may want to tackle it as a school project. Here are the two major aspects I’ve identified. A general scraper for calendar-like web pages There are zillions of calendar-like web pages, like this one for Harlow’s Pub in Peterborough, NH. These ideally ought to be maintained using calendar programs that publish machine-readable iCalendar feeds which are also transformed and styled to create human-readable web pages. But that doesn’t (yet) commonly happen. These web pages are, however, often amenable to scraping. And for a while, elmcity curators were making very effective use of FuseCal (1, 2, 3) to transform these kinds of pages into iCalendar feeds. When that service shut down, I retained a list of the pages that elmcity curators were successfully transforming into iCalendar feeds using FuseCal. These are test cases for an HTML-to-iCalendar service. Anyone who’s handy with scraping libraries like Beautiful Soup can solve these individually. The challenge here is to create, by abstraction and generalization, an engine that can handle a significant swath of these cases. A hybrid system for finding implicit recurring events and making them explicit Lots of implicit calendar data online doesn’t even pretend to be calendar-like, and cannot be harvested using a scraper. Finding one-off events in this category is out of scope for my project. But finding recurring events seems promising. The singular effort required to publish one of these will pay ongoing dividends. It’s helpful that the language people use to describe these events — “every Tuesday”, “third Saturday of every month” — is distinctive. To being exploring this domain, I wrote a specialized search robot that looks for these patterns, in conjunction with names of places. Its output is available for all the cities and towns participating in the elmcity project. For example, this page is the output for Keene, NH. It includes more than 2000 links to web pages — or, quite often, PDF files — some fraction of which represent recurring events. In Finding and connecting social capital I showed a couple of cases where the pages found this way did, in fact, represent recurring events that could be added to an iCalendar feed. To a computer scientist this looks like a problem that you might solve using a natural language parser. And I think it is partly that, but only partly. Let’s look at another example: At first glance, this looks hopeful: First Monday of each month: Dads Group, 105 Castle Street, Keene NH But the real world is almost always messier than that. For starters, that image comes from the Monadnock Men’s Resource Center’s Fall 2004 newsletter. So before I add this to a calendar, I’ll want to confirm the information. The newsletter is hosted at the MMRC site. Investigation yields these observations: The most recent issue of the newsletter was Winter ‘06 The last-modified date of the MMRC home page is September 2008 As of that date, the Dads Group still seems to have been active, under a slightly different name: Parent Outreach Project, DadTime Program, 355-3082 There’s no email address, only a phone number. So I called the number, left a message, and will soon know the current status. What kind of software-based system can help us scale this gnarly process? There is an algorithmic solution, surely, but it will need to operate in a hybrid environment. The initial search-driven discovery of candidate events can be done by an automated parser tuned for this domain. But the verification of candidates will need to be done by human volunteers, assisted by software that helps them: Divide long lists of candidates into smaller batches Work in parallel on those batches Evaluate the age and provenance of candidates Verify or disqualify candidates based on discoverable evidence, if possible Otherwise, find appropriate email addresses (preferably) or phone numbers, and manage the back-and-forth communication required to verify or disqualify a candidate Refer event sponsors to a calendar publishing how-to, and invite them to create data feeds that can reliably syndicate Students endowed with the geek gene are likely to gravitate toward the first problem because it’s cleaner. But I hope I can also attract interest in the second problem. We really need people who can hack that kind of real-world messiness. tag:odeo.com,2009-08-10,24938744 Mon, 10 Aug 2009 10:18:44 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24926000-That-word-%E2%80%9Cevents%E2%80%9D-It-does-not-mean-what-you-think-it-means In of one of my favorite scenes from one my favorite movies, The Princess Bride, Vizzini (Wallace Shawn) has been repeatedly exclaiming: “Inconceivable!” Finally Inigo Montoya (Mandy Patinkin) responds: You keep using that word. I do not think it means what you think it means. I’ve already riffed on that classic bit in the titles of two other items. Now I’m compelled to do it again because when I talk about events, vis-a-vis the elmcity project, I think the word means something different from you probably think it means. Here’s one common meaning: major public events. These include things like artistic performances, festivals, fairs, and sporting events. They dominate the “Things to See and Do” section of every newspaper and online community guide, and are usually well publicized. Here’s another common meaning: minor events that are often (but not aways) private. These include birthday parties, house concerts, and outdoor excursions. T... In of one of my favorite scenes from one my favorite movies, The Princess Bride, Vizzini (Wallace Shawn) has been repeatedly exclaiming: “Inconceivable!” Finally Inigo Montoya (Mandy Patinkin) responds: You keep using that word. I do not think it means what you think it means. I’ve already riffed on that classic bit in the titles of two other items. Now I’m compelled to do it again because when I talk about events, vis-a-vis the elmcity project, I think the word means something different from you probably think it means. Here’s one common meaning: major public events. These include things like artistic performances, festivals, fairs, and sporting events. They dominate the “Things to See and Do” section of every newspaper and online community guide, and are usually well publicized. Here’s another common meaning: minor events that are often (but not aways) private. These include birthday parties, house concerts, and outdoor excursions. They are, nowadays, often publicized very well in Facebook. Although I’m happy to see major public events showing up in an elmcity hub, that isn’t my main goal. And private events, of course, don’t belong in an elmcity hub, they belong in Facebook, or in other private networks. There’s a third kind of event that interests me most of all. It occupies a space between the other two. It’s public, but minor: a book discussion, a roadside cleanup, a support group, a squaredance. These events typically don’t show up in “Things To See And Do” guides because they’re considered too niche, and because it’s too much work — for both the publisher and the contributor — to get them included. They might show up in Facebook, but if so they will be visible there only within a closed social network. There are tons of events in this minor-but-public category. Here’s one of my favorite examples. We were having dinner with our friends Lin and Tom recently, and Lin mentioned that Tom had just won the New Hampshire state archery tournament. Me: “Really? Congratulations! Where was that held?” Lin: “At the Keene Recreation Center, last Saturday.” The Rec Center is a ten-minute walk from my house. I’d have loved to have seen those precision archers ply their trade. And it was open to the public. Anybody could have gone. But nobody knew. Everyone I talk to has similar stories. Everyone says they find out about such things — if they find out at all — only after the fact. Everyone acknowledges that there should be a better way to inform one another about the goings-on that implicitly form much of the social capital of the community. If we can make more of it explicit, we will lead richer lives. And here I mean richer in two senses of that word. There’s the Robert Putnam sense of social well-being. And there’s the Richard Florida sense of economic well-being. If we can make more of our implicit social capital explicit, we’ll profit in both ways. In of one of my favorite scenes from one my favorite movies, The Princess Bride, Vizzini (Wallace Shawn) has been repeatedly exclaiming: “Inconceivable!” Finally Inigo Montoya (Mandy Patinkin) responds: You keep using that word. I do not think it means what you think it means. I’ve already riffed on that classic bit in the titles of two other items. Now I’m compelled to do it again because when I talk about events, vis-a-vis the elmcity project, I think the word means something different from you probably think it means. Here’s one common meaning: major public events. These include things like artistic performances, festivals, fairs, and sporting events. They dominate the “Things to See and Do” section of every newspaper and online community guide, and are usually well publicized. Here’s another common meaning: minor events that are often (but not aways) private. These include birthday parties, house concerts, and outdoor excursions. They are, nowadays, often publicized very well in Facebook. Although I’m happy to see major public events showing up in an elmcity hub, that isn’t my main goal. And private events, of course, don’t belong in an elmcity hub, they belong in Facebook, or in other private networks. There’s a third kind of event that interests me most of all. It occupies a space between the other two. It’s public, but minor: a book discussion, a roadside cleanup, a support group, a squaredance. These events typically don’t show up in “Things To See And Do” guides because they’re considered too niche, and because it’s too much work — for both the publisher and the contributor — to get them included. They might show up in Facebook, but if so they will be visible there only within a closed social network. There are tons of events in this minor-but-public category. Here’s one of my favorite examples. We were having dinner with our friends Lin and Tom recently, and Lin mentioned that Tom had just won the New Hampshire state archery tournament. Me: “Really? Congratulations! Where was that held?” Lin: “At the Keene Recreation Center, last Saturday.” The Rec Center is a ten-minute walk from my house. I’d have loved to have seen those precision archers ply their trade. And it was open to the public. Anybody could have gone. But nobody knew. Everyone I talk to has similar stories. Everyone says they find out about such things — if they find out at all — only after the fact. Everyone acknowledges that there should be a better way to inform one another about the goings-on that implicitly form much of the social capital of the community. If we can make more of it explicit, we will lead richer lives. And here I mean richer in two senses of that word. There’s the Robert Putnam sense of social well-being. And there’s the Richard Florida sense of economic well-being. If we can make more of our implicit social capital explicit, we’ll profit in both ways. tag:odeo.com,2009-08-07,24926000 Fri, 07 Aug 2009 09:33:54 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24914348-Curation-meta-curation-and-live-Net-radio I’ve long been dissatisfied with how we discover and tune into Net radio. This iTunes screenshot illustrates the problem: Start with a genre, pick a station in that genre, then listen to that station. This just doesn’t work for me. I like to listen to a lot of different things. And I especially value serendipitous recommendations from curators whose knowledge and preferences diverge radically from my own. Yes there’s Pandora, but what I’ve been wanting all along is a way to enable and then subscribe to curators who guide me to what’s playing now on the live streams coming from radio stations around the world. It’s Wednesday morning, 11AM Eastern Daylight Time, and I know there are all kinds of shows playing right now. But how do I materialize a view for this moment in time — or for tonight at 9PM, or for Sunday morning at 10AM — across that breadth and wealth of live streams? I started thinking about schedules of radio programs, and ab... I’ve long been dissatisfied with how we discover and tune into Net radio. This iTunes screenshot illustrates the problem: Start with a genre, pick a station in that genre, then listen to that station. This just doesn’t work for me. I like to listen to a lot of different things. And I especially value serendipitous recommendations from curators whose knowledge and preferences diverge radically from my own. Yes there’s Pandora, but what I’ve been wanting all along is a way to enable and then subscribe to curators who guide me to what’s playing now on the live streams coming from radio stations around the world. It’s Wednesday morning, 11AM Eastern Daylight Time, and I know there are all kinds of shows playing right now. But how do I materialize a view for this moment in time — or for tonight at 9PM, or for Sunday morning at 10AM — across that breadth and wealth of live streams? I started thinking about schedules of radio programs, and about calendars, and about BBC Backstage — because I’ll be interviewing Ian Forrester for an upcoming episode of my podcast — and I landed on this blog post which shows how to form an URL that retrieves upcoming episodes of a BBC show as an iCalendar feed. Meanwhile, I’ve just created a new mode for the elmcity calendar aggregator. Now instead of creating a geographical hub, which combines events from Eventful and Upcoming and events from a list of iCalendar feeds — all for one location — you can create a topical hub whose events are governed only by time, not by location. Can these ingredients combine to solve my Net radio problem? Could a curator for an elmcity topical aggregator cherrypick favorite shows from around the Net, and create a calendar that shows me what’s playing right now? It seems plausible, so I spun up a new topical hub in the elmcity aggregator and started experimenting. I started by looking at the BBC’s iCalendar feeds. For a given show, you can form a variant of its URL that retrieves upcoming shows in iCalendar format. But evidently these iCalendar feeds don’t include VTIMEZONE components, which means calendar clients (or aggregators) can’t translate UK times to other times. I ran into a few other issues, which perhaps can be sorted out when I chat with Ian Forrester. But meanwhile, since the universe of Net radio is much vaster than the BBC, and since most of it won’t be accessible in the form of data feeds, I stepped back for a broader view. Really, anyone can publish an event that gives the time for a live show, plus a link to its player. And when a show happens on a regular recurring schedule, the little bit of effort it takes to publish that event pays recurring dividends. Consider, for example, Nic Harcourt’s Sounds Eclectic. It’s on at these (Pacific) times: SUN 6:00A-8:00A, SAT 2:00P-4:00P, SAT 10:00P-12:00A. You can plug these into any calendar program as recurring events. And if you publish a feed, it’s not only available to you from any calendar client, it’s also available to any other calendar client — or to any aggregator. Here’s a calendar with three recurring events for Sounds Eclectic, plus one recurring event for WICN’s Sunday jazz show, plus a single non-recurring event — the BBC’s Folkscene — which will be on the BBC iPlayer on Thursday at 4:05PM my time and 9:05PM UK time. If you load the calendar feed into a client — Outlook, Apple iCal, Google Calendar, Lotus Notes — you’ll see these events translated into your local timezone. Note that Live Calendar is especially handy for publishing events from many different timezones. That’s because like Outlook, but unlike Google Calendar, it enables you to specify timezones on a per-event basis. So instead of having to enter the Sunday morning recurrence of Sounds Eclectic as 9AM Eastern Daylight, I can enter it as 6AM Pacific Daylight Time. Likewise Folkscene: I can enter 9:05 British Summer Time. This sort of calendar is great for personal use. But I’m looking for the Webjay of Net radio. And I think maybe elmcity topical hubs can help enable that. There’s a way of using these topical hubs I hadn’t thought of until Tony Karrer created one. Tony runs TechEmpower, a software, web, and eLearning development firm. He wants to track and publish online eLearning events, so he’s managing them in Google Calendar and syndicating them through an elmcity topical hub to his website. A topical hub, like a geographic hub, is controlled by a Delicious account whose owner maintains a list of feeds. I’d been thinking of the account owner as the curator, and of the feeds as homogeneous sources of events: school board meetings, soccer games, and so on. But then Tony partnered with another organization that tracks webinars, invited that group to publish its own feed, added it to the eLearning hub, and wrote a blog post entitled Second Calendar Curator Joins to Help with List of Free Webinars: The initial list of calendar entries, we added ourselves. But I’m pleased to announce that we’ve just signed up our second calendar curator – Coaching Ourselves. Their events are now appearing in the listings. … It is exactly because we can distribute the load of keeping this list current that makes me think this will work really well in the long run. This probably shouldn’t have surprised me, but it did. I’d been thinking in terms of curators, feeds, and events. What Tony showed me is that you can also (optionally) think in terms of meta-curators, curators, feeds, and events. In this example, Tony is himself a curator, but he is also a meta-curator — that is, a collector of curators. I’d love to see this model evolve in the realm of Net radio. If you want to join the experiment, just use any calendar program to keep track of some of your favorite recurring shows. (Again, it’s very helpful to use one that supports per-event timezones.) Then publish the shows as an iCalendar feed, and send me the URL. As the meta-curator of delicious.com/InternetRadio, as well as the curator of jonu.calendar.live.com/calendar/InternetRadio/index.html, I’ll have two options. If I like most or all of the shows you like, I can add your feed to the hub. If I only like some of the shows you like, I can cherrypick them for my feed. Either way, the aggregated results will be available as XML, as JSON, and as an iCalendar feed that can flow into calendar clients or aggregators. Naturally there can also be other meta-curators. To become one, designate a Delicious account for the purpose, spin up your own topical hub, and tell me about it. I’ve long been dissatisfied with how we discover and tune into Net radio. This iTunes screenshot illustrates the problem: Start with a genre, pick a station in that genre, then listen to that station. This just doesn’t work for me. I like to listen to a lot of different things. And I especially value serendipitous recommendations from curators whose knowledge and preferences diverge radically from my own. Yes there’s Pandora, but what I’ve been wanting all along is a way to enable and then subscribe to curators who guide me to what’s playing now on the live streams coming from radio stations around the world. It’s Wednesday morning, 11AM Eastern Daylight Time, and I know there are all kinds of shows playing right now. But how do I materialize a view for this moment in time — or for tonight at 9PM, or for Sunday morning at 10AM — across that breadth and wealth of live streams? I started thinking about schedules of radio programs, and about calendars, and about BBC Backstage — because I’ll be interviewing Ian Forrester for an upcoming episode of my podcast — and I landed on this blog post which shows how to form an URL that retrieves upcoming episodes of a BBC show as an iCalendar feed. Meanwhile, I’ve just created a new mode for the elmcity calendar aggregator. Now instead of creating a geographical hub, which combines events from Eventful and Upcoming and events from a list of iCalendar feeds — all for one location — you can create a topical hub whose events are governed only by time, not by location. Can these ingredients combine to solve my Net radio problem? Could a curator for an elmcity topical aggregator cherrypick favorite shows from around the Net, and create a calendar that shows me what’s playing right now? It seems plausible, so I spun up a new topical hub in the elmcity aggregator and started experimenting. I started by looking at the BBC’s iCalendar feeds. For a given show, you can form a variant of its URL that retrieves upcoming shows in iCalendar format. But evidently these iCalendar feeds don’t include VTIMEZONE components, which means calendar clients (or aggregators) can’t translate UK times to other times. I ran into a few other issues, which perhaps can be sorted out when I chat with Ian Forrester. But meanwhile, since the universe of Net radio is much vaster than the BBC, and since most of it won’t be accessible in the form of data feeds, I stepped back for a broader view. Really, anyone can publish an event that gives the time for a live show, plus a link to its player. And when a show happens on a regular recurring schedule, the little bit of effort it takes to publish that event pays recurring dividends. Consider, for example, Nic Harcourt’s Sounds Eclectic. It’s on at these (Pacific) times: SUN 6:00A-8:00A, SAT 2:00P-4:00P, SAT 10:00P-12:00A. You can plug these into any calendar program as recurring events. And if you publish a feed, it’s not only available to you from any calendar client, it’s also available to any other calendar client — or to any aggregator. Here’s a calendar with three recurring events for Sounds Eclectic, plus one recurring event for WICN’s Sunday jazz show, plus a single non-recurring event — the BBC’s Folkscene — which will be on the BBC iPlayer on Thursday at 4:05PM my time and 9:05PM UK time. If you load the calendar feed into a client — Outlook, Apple iCal, Google Calendar, Lotus Notes — you’ll see these events translated into your local timezone. Note that Live Calendar is especially handy for publishing events from many different timezones. That’s because like Outlook, but unlike Google Calendar, it enables you to specify timezones on a per-event basis. So instead of having to enter the Sunday morning recurrence of Sounds Eclectic as 9AM Eastern Daylight, I can enter it as 6AM Pacific Daylight Time. Likewise Folkscene: I can enter 9:05 British Summer Time. This sort of calendar is great for personal use. But I’m looking for the Webjay of Net radio. And I think maybe elmcity topical hubs can help enable that. There’s a way of using these topical hubs I hadn’t thought of until Tony Karrer created one. Tony runs TechEmpower, a software, web, and eLearning development firm. He wants to track and publish online eLearning events, so he’s managing them in Google Calendar and syndicating them through an elmcity topical hub to his website. A topical hub, like a geographic hub, is controlled by a Delicious account whose owner maintains a list of feeds. I’d been thinking of the account owner as the curator, and of the feeds as homogeneous sources of events: school board meetings, soccer games, and so on. But then Tony partnered with another organization that tracks webinars, invited that group to publish its own feed, added it to the eLearning hub, and wrote a blog post entitled Second Calendar Curator Joins to Help with List of Free Webinars: The initial list of calendar entries, we added ourselves. But I’m pleased to announce that we’ve just signed up our second calendar curator – Coaching Ourselves. Their events are now appearing in the listings. … It is exactly because we can distribute the load of keeping this list current that makes me think this will work really well in the long run. This probably shouldn’t have surprised me, but it did. I’d been thinking in terms of curators, feeds, and events. What Tony showed me is that you can also (optionally) think in terms of meta-curators, curators, feeds, and events. In this example, Tony is himself a curator, but he is also a meta-curator — that is, a collector of curators. I’d love to see this model evolve in the realm of Net radio. If you want to join the experiment, just use any calendar program to keep track of some of your favorite recurring shows. (Again, it’s very helpful to use one that supports per-event timezones.) Then publish the shows as an iCalendar feed, and send me the URL. As the meta-curator of delicious.com/InternetRadio, as well as the curator of jonu.calendar.live.com/calendar/InternetRadio/index.html, I’ll have two options. If I like most or all of the shows you like, I can add your feed to the hub. If I only like some of the shows you like, I can cherrypick them for my feed. Either way, the aggregated results will be available as XML, as JSON, and as an iCalendar feed that can flow into calendar clients or aggregators. Naturally there can also be other meta-curators. To become one, designate a Delicious account for the purpose, spin up your own topical hub, and tell me about it. tag:odeo.com,2009-08-05,24914348 Wed, 05 Aug 2009 08:50:10 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24903734-Talking-with-Cathy-Marshall-about-tags-digital-archiving-and-lifestreams My guest for this week’s Innovators show is Cathy Marshall, a Senior Researcher in Microsoft’s Silicon Valley Lab. She’s long been intrigued by personal information management — and nowadays, also by its social dimension. We kicked off the conversation with a discussion of her recent paper Do Tags Work?. (See also her slides from a talk about the project.) This was a clever study in which she collected a bunch of Flickr photos of people spinning on the bull’s balls in Milan. Notice how that fulltext query effectively retrieves a pile of images, taken by different people, of the same curious custom: If you are passing through the Galleria Vittorio Emanuele II, you should spin around on the testicles of the bull mosaic found in the centre. Legend has it that this will bring you good luck! Now try this query, which uses the same terms but looks at tags instead of the free text (title, description) associated with the photos. It finds nothing. Cathy conclud... My guest for this week’s Innovators show is Cathy Marshall, a Senior Researcher in Microsoft’s Silicon Valley Lab. She’s long been intrigued by personal information management — and nowadays, also by its social dimension. We kicked off the conversation with a discussion of her recent paper Do Tags Work?. (See also her slides from a talk about the project.) This was a clever study in which she collected a bunch of Flickr photos of people spinning on the bull’s balls in Milan. Notice how that fulltext query effectively retrieves a pile of images, taken by different people, of the same curious custom: If you are passing through the Galleria Vittorio Emanuele II, you should spin around on the testicles of the bull mosaic found in the centre. Legend has it that this will bring you good luck! Now try this query, which uses the same terms but looks at tags instead of the free text (title, description) associated with the photos. It finds nothing. Cathy concludes that while many people think tags are effective hooks for information retrieval, they really aren’t. Of course, those of us who attend conferences where the first order of business is to announce a tag know that tags can be a very effective way to aggregate all the blog postings, tweets, and photos associated with an event. Folksonomies that aren’t intended to converge don’t. Those that are meant to converge do, quite dramatically, which is why I’ve long been obsessed with intentional tagging as an enabler of loosely-coupled collaboration. In the second half of the conversation we discussed personal digital archiving, curation, benign neglect, and lifestreams. Cathy tells a lot of stories about the ways in which people do, and also don’t, take care of their digital stuff. She observes, for example, that when people lose the contents of a computer, they react initially with horror, but then often feel a sense of relief. It turns out a lot of what was there wasn’t really needed. The burden of culling through it is lifted, and the guilt associated with not doing that culling that goes away. (I laughed harder than I have in a long time when Cathy described rental storage units as “garbage cans you pay for, and then when you realize you no longer care about the stuff in them, you stop paying for.”) We ended by agreeing that the hardest thing about introducing a hosted lifebits service ecosystem will be the conceptual model. For psychological reasons, people will want to think in terms of monolithic containers that keep stuff in one place, and monolithic services that do everything related to that stuff. For architectural reasons, though, we’ll want to federate storage, and also decouple classes of service — so that storage, for example, is orthogonal to access control and authorization, which is orthogonal to social interaction. My guest for this week’s Innovators show is Cathy Marshall, a Senior Researcher in Microsoft’s Silicon Valley Lab. She’s long been intrigued by personal information management — and nowadays, also by its social dimension. We kicked off the conversation with a discussion of her recent paper Do Tags Work?. (See also her slides from a talk about the project.) This was a clever study in which she collected a bunch of Flickr photos of people spinning on the bull’s balls in Milan. Notice how that fulltext query effectively retrieves a pile of images, taken by different people, of the same curious custom: If you are passing through the Galleria Vittorio Emanuele II, you should spin around on the testicles of the bull mosaic found in the centre. Legend has it that this will bring you good luck! Now try this query, which uses the same terms but looks at tags instead of the free text (title, description) associated with the photos. It finds nothing. Cathy concludes that while many people think tags are effective hooks for information retrieval, they really aren’t. Of course, those of us who attend conferences where the first order of business is to announce a tag know that tags can be a very effective way to aggregate all the blog postings, tweets, and photos associated with an event. Folksonomies that aren’t intended to converge don’t. Those that are meant to converge do, quite dramatically, which is why I’ve long been obsessed with intentional tagging as an enabler of loosely-coupled collaboration. In the second half of the conversation we discussed personal digital archiving, curation, benign neglect, and lifestreams. Cathy tells a lot of stories about the ways in which people do, and also don’t, take care of their digital stuff. She observes, for example, that when people lose the contents of a computer, they react initially with horror, but then often feel a sense of relief. It turns out a lot of what was there wasn’t really needed. The burden of culling through it is lifted, and the guilt associated with not doing that culling that goes away. (I laughed harder than I have in a long time when Cathy described rental storage units as “garbage cans you pay for, and then when you realize you no longer care about the stuff in them, you stop paying for.”) We ended by agreeing that the hardest thing about introducing a hosted lifebits service ecosystem will be the conceptual model. For psychological reasons, people will want to think in terms of monolithic containers that keep stuff in one place, and monolithic services that do everything related to that stuff. For architectural reasons, though, we’ll want to federate storage, and also decouple classes of service — so that storage, for example, is orthogonal to access control and authorization, which is orthogonal to social interaction. tag:odeo.com,2009-08-03,24903734 Mon, 03 Aug 2009 07:12:09 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24892235-Polymath-user-innovation In February 2007, Mike Adams, who had recently joined Automattic, the company that makes WordPress, decided on a lark to endow all blogs running on WordPress.com with the ability to use LaTeX, the venerable mathematical typesetting language. So I can write this: $latex \pi r^2$ And produce this: When he introduced the feature, Mike wrote: Odd as it may sound, I miss all the equations from my days in grad school, so I decided that what WordPress.com needed most was a hot, niche feature that maybe 17 people would use regularly. A whole lot more than 17 people cared. And some of them, it turns out, are Fields medalists. Back in January, one member of that elite group — Tim Gowers — asked: Is massively collaborative mathematics possible? Since then, as reported by observer/participant Michael Nielsen (1, 2), Tim Gowers, Terence Tao, and a bunch of their peers have been pioneering a massively collaborative approach to solving hard mathematical problems. Reflecting on ... In February 2007, Mike Adams, who had recently joined Automattic, the company that makes WordPress, decided on a lark to endow all blogs running on WordPress.com with the ability to use LaTeX, the venerable mathematical typesetting language. So I can write this: $latex \pi r^2$ And produce this: When he introduced the feature, Mike wrote: Odd as it may sound, I miss all the equations from my days in grad school, so I decided that what WordPress.com needed most was a hot, niche feature that maybe 17 people would use regularly. A whole lot more than 17 people cared. And some of them, it turns out, are Fields medalists. Back in January, one member of that elite group — Tim Gowers — asked: Is massively collaborative mathematics possible? Since then, as reported by observer/participant Michael Nielsen (1, 2), Tim Gowers, Terence Tao, and a bunch of their peers have been pioneering a massively collaborative approach to solving hard mathematical problems. Reflecting on the outcome of the first polymath experiment, Michael Nielsen wrote: The scope of participation in the project is remarkable. More than 1000 mathematical comments have been written on Gowers’ blog, and the blog of Terry Tao, another mathematician who has taken a leading role in the project. The Polymath wiki has approximately 59 content pages, with 11 registered contributors, and more anonymous contributors. It’s already a remarkable resource on the density Hales-Jewett theorem and related topics. The project timeline shows notable mathematical contributions being made by 23 contributors to date. This was accomplished in seven weeks. Just this week, a polymath blog has emerged to serve as an online home for the further evolution of this approach. I am completely unqualified to evaluate the nature of mathematical discourse that’s going in on these polymath collaborations, or the claims being made regarding outcomes. But it sure makes my spidey-sense tingle. I am, however, qualified to evaluate the nature of the collaborative methods being employed. And on that front, I’m amused (and chagrined) to recall something I wrote back in 2000, in a report called Internet groupware for scientific collaboration. The report was commissioned by Greg Wilson, who organized this week’s Science 2.0 event in Toronto. At that event, my report served as a historical frame for the polymath experimentation that’s going on right now, and that Michael Nielsen discussed at the Toronto event in an updated version of this talk. In my 2000 report I said: TeX and LaTeX define scientific publishing for a generation of scientists. But these formats don’t integrate directly into the shared spaces of the Web. The rise of XML as a universal markup language, along with vocabularies such as MathML (for mathematical notation) and SVG (for scalable vector graphics), suggests that the Web may yet reach its original collaborative goal. Why didn’t I see, then, that the crux of the issue wasn’t XML and MathML and SVG, but rather the ability to “integrate directly into the shared spaces of the Web”? And that what ought to be integrated directly was the typesetting language already familiar to mathematicians, namely LaTeX? The answer is that I needed (and still need) to be reminded that good-enough solutions here now, and familiar to people, often trump great solutions that aren’t here and wouldn’t be familiar if they were. From that perspective, I’m wondering what will and won’t turn out be good enough for the polymathematicians. The current setup is admittedly imperfect, and they’re now begining to explore WordPress plugins that enable, for example, more powerful ways to organize, reply to, and refer to one anothers’ comments. I don’t think anybody yet knows what the right tooling will be for polymathematical collaboration. The ones who are best qualified to figure it out are the polymathematical collaborators themselves, but they are not WordPress plugin developers. What’s needed is what Eric von Hippel calls a user innovation toolkit. The idea is this: Leading users, as they employ a tool, also modify it, and in so doing they express intentions that tool developers can then capture and formalize. If you look at the systems of notation that the polymathematicians are creating in order to organize and refer to their contributions in these long and complex threads of mathematical discourse, you can see intentions being expressed. So arguably, WordPress is a user innovation toolkit, and we’ll see these innovations codified in future plugins. I’ll be watching with great interest. In February 2007, Mike Adams, who had recently joined Automattic, the company that makes WordPress, decided on a lark to endow all blogs running on WordPress.com with the ability to use LaTeX, the venerable mathematical typesetting language. So I can write this: $latex \pi r^2$ And produce this: When he introduced the feature, Mike wrote: Odd as it may sound, I miss all the equations from my days in grad school, so I decided that what WordPress.com needed most was a hot, niche feature that maybe 17 people would use regularly. A whole lot more than 17 people cared. And some of them, it turns out, are Fields medalists. Back in January, one member of that elite group — Tim Gowers — asked: Is massively collaborative mathematics possible? Since then, as reported by observer/participant Michael Nielsen (1, 2), Tim Gowers, Terence Tao, and a bunch of their peers have been pioneering a massively collaborative approach to solving hard mathematical problems. Reflecting on the outcome of the first polymath experiment, Michael Nielsen wrote: The scope of participation in the project is remarkable. More than 1000 mathematical comments have been written on Gowers’ blog, and the blog of Terry Tao, another mathematician who has taken a leading role in the project. The Polymath wiki has approximately 59 content pages, with 11 registered contributors, and more anonymous contributors. It’s already a remarkable resource on the density Hales-Jewett theorem and related topics. The project timeline shows notable mathematical contributions being made by 23 contributors to date. This was accomplished in seven weeks. Just this week, a polymath blog has emerged to serve as an online home for the further evolution of this approach. I am completely unqualified to evaluate the nature of mathematical discourse that’s going in on these polymath collaborations, or the claims being made regarding outcomes. But it sure makes my spidey-sense tingle. I am, however, qualified to evaluate the nature of the collaborative methods being employed. And on that front, I’m amused (and chagrined) to recall something I wrote back in 2000, in a report called Internet groupware for scientific collaboration. The report was commissioned by Greg Wilson, who organized this week’s Science 2.0 event in Toronto. At that event, my report served as a historical frame for the polymath experimentation that’s going on right now, and that Michael Nielsen discussed at the Toronto event in an updated version of this talk. In my 2000 report I said: TeX and LaTeX define scientific publishing for a generation of scientists. But these formats don’t integrate directly into the shared spaces of the Web. The rise of XML as a universal markup language, along with vocabularies such as MathML (for mathematical notation) and SVG (for scalable vector graphics), suggests that the Web may yet reach its original collaborative goal. Why didn’t I see, then, that the crux of the issue wasn’t XML and MathML and SVG, but rather the ability to “integrate directly into the shared spaces of the Web”? And that what ought to be integrated directly was the typesetting language already familiar to mathematicians, namely LaTeX? The answer is that I needed (and still need) to be reminded that good-enough solutions here now, and familiar to people, often trump great solutions that aren’t here and wouldn’t be familiar if they were. From that perspective, I’m wondering what will and won’t turn out be good enough for the polymathematicians. The current setup is admittedly imperfect, and they’re now begining to explore WordPress plugins that enable, for example, more powerful ways to organize, reply to, and refer to one anothers’ comments. I don’t think anybody yet knows what the right tooling will be for polymathematical collaboration. The ones who are best qualified to figure it out are the polymathematical collaborators themselves, but they are not WordPress plugin developers. What’s needed is what Eric von Hippel calls a user innovation toolkit. The idea is this: Leading users, as they employ a tool, also modify it, and in so doing they express intentions that tool developers can then capture and formalize. If you look at the systems of notation that the polymathematicians are creating in order to organize and refer to their contributions in these long and complex threads of mathematical discourse, you can see intentions being expressed. So arguably, WordPress is a user innovation toolkit, and we’ll see these innovations codified in future plugins. I’ll be watching with great interest. tag:odeo.com,2009-07-31,24892235 Fri, 31 Jul 2009 10:10:21 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24869819-Talking-with-Mike-Dunn-about-practical-uses-of-semantic-technology My guest for this week’s Innovators show is Mike Dunn, a veteran media technologist who recently attended, and spoke at, the 2009 Semantic Technology. Mike and I were both impressed by Tom Tague’s keynote talk, which avoided theory and focused on practical ways that here-and-now semantic technologies are helping media businesses work smarter and more profitably. In this conversation, Mike describes some of the ways that his company, Hearst Media Interactive, is proving that point. Search engine optimization is currently one of the best ways to profit from data-enabled content. Meanwhile, one of the expected benefits of semantic technology — better search recall and precision — hasn’t materialized. But although most users may not care about querying archives more comprehensively and more precisely, writers and editors should. And not only because it helps automate the assembly of context around a current story. If you can review an archive in a precise a... My guest for this week’s Innovators show is Mike Dunn, a veteran media technologist who recently attended, and spoke at, the 2009 Semantic Technology. Mike and I were both impressed by Tom Tague’s keynote talk, which avoided theory and focused on practical ways that here-and-now semantic technologies are helping media businesses work smarter and more profitably. In this conversation, Mike describes some of the ways that his company, Hearst Media Interactive, is proving that point. Search engine optimization is currently one of the best ways to profit from data-enabled content. Meanwhile, one of the expected benefits of semantic technology — better search recall and precision — hasn’t materialized. But although most users may not care about querying archives more comprehensively and more precisely, writers and editors should. And not only because it helps automate the assembly of context around a current story. If you can review an archive in a precise and comprehensive way, you can do a better job of planning future stories that acknowledge — and advance — the ones you’ve already done. My guest for this week’s Innovators show is Mike Dunn, a veteran media technologist who recently attended, and spoke at, the 2009 Semantic Technology. Mike and I were both impressed by Tom Tague’s keynote talk, which avoided theory and focused on practical ways that here-and-now semantic technologies are helping media businesses work smarter and more profitably. In this conversation, Mike describes some of the ways that his company, Hearst Media Interactive, is proving that point. Search engine optimization is currently one of the best ways to profit from data-enabled content. Meanwhile, one of the expected benefits of semantic technology — better search recall and precision — hasn’t materialized. But although most users may not care about querying archives more comprehensively and more precisely, writers and editors should. And not only because it helps automate the assembly of context around a current story. If you can review an archive in a precise and comprehensive way, you can do a better job of planning future stories that acknowledge — and advance — the ones you’ve already done. tag:odeo.com,2009-07-27,24869819 Mon, 27 Jul 2009 07:42:18 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24859169-Topical-event-hubs The elmcity project began with a focus on aggregating events for communities defined by places: cities, towns. But I realized a while ago that it could also be used to aggregate events for communities defined by topics. So now I’m building out that capability. One early adopter tracks and promotes online events in the e-learning domain. Another tracks and promotes conferences and events related to environmentally-sustainable business practices. The curation method is very similar to what’s defined in the elmcity project FAQ. To define a topic hub you use a Delicious account, you create a metadata URL as shown in the FAQ, and you use what= instead of where= to define a topic instead of a location. Since there’s no location, there’s no aggregation of Eventful and Upcoming events. The topical hub is driven purely by your registry of iCalendar feeds. If you (or somebody you know) needs to curate events by topic, and would like try this method, please get in touch... The elmcity project began with a focus on aggregating events for communities defined by places: cities, towns. But I realized a while ago that it could also be used to aggregate events for communities defined by topics. So now I’m building out that capability. One early adopter tracks and promotes online events in the e-learning domain. Another tracks and promotes conferences and events related to environmentally-sustainable business practices. The curation method is very similar to what’s defined in the elmcity project FAQ. To define a topic hub you use a Delicious account, you create a metadata URL as shown in the FAQ, and you use what= instead of where= to define a topic instead of a location. Since there’s no location, there’s no aggregation of Eventful and Upcoming events. The topical hub is driven purely by your registry of iCalendar feeds. If you (or somebody you know) needs to curate events by topic, and would like try this method, please get in touch. I’d love to have you help me define how this can work, and discover where it can go. The elmcity project began with a focus on aggregating events for communities defined by places: cities, towns. But I realized a while ago that it could also be used to aggregate events for communities defined by topics. So now I’m building out that capability. One early adopter tracks and promotes online events in the e-learning domain. Another tracks and promotes conferences and events related to environmentally-sustainable business practices. The curation method is very similar to what’s defined in the elmcity project FAQ. To define a topic hub you use a Delicious account, you create a metadata URL as shown in the FAQ, and you use what= instead of where= to define a topic instead of a location. Since there’s no location, there’s no aggregation of Eventful and Upcoming events. The topical hub is driven purely by your registry of iCalendar feeds. If you (or somebody you know) needs to curate events by topic, and would like try this method, please get in touch. I’d love to have you help me define how this can work, and discover where it can go. tag:odeo.com,2009-07-24,24859169 Fri, 24 Jul 2009 12:18:27 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24851039-Why-we-need-an-XML-representation-for-iCalendar On this week’s Innovators show I got together with two of the authors of a new proposal for representing iCalendar in XML. Mike Douglass is lead developer of the Bedework Calendar System, and Steven Lees is Microsoft’s program manager for FeedSync and chair of the XML technical committee in CalConnect, the Calendaring and Scheduling Consortium. What’s proposed is no more, but no less, than a well-defined two-way mapping between the current non-XML-based iCalendar format and an equivalent XML format. So, for example, here’s an event — the first low tide of 2009 in Myrtle Beach, SC — in iCalendar format: BEGIN:VEVENT SUMMARY:Low Tide 0.39 ft DTSTART:20090101T090000Z UID:2009.0 DTSTAMP:20080527T000001Z END:VEVENT And here’s the equivalent XML: <vevent> <properties> <dtstamp> <date-time utc='yes'> <year>2008</year><month>5</month><day>27</day> <hour>0</hour><minute>0&l... On this week’s Innovators show I got together with two of the authors of a new proposal for representing iCalendar in XML. Mike Douglass is lead developer of the Bedework Calendar System, and Steven Lees is Microsoft’s program manager for FeedSync and chair of the XML technical committee in CalConnect, the Calendaring and Scheduling Consortium. What’s proposed is no more, but no less, than a well-defined two-way mapping between the current non-XML-based iCalendar format and an equivalent XML format. So, for example, here’s an event — the first low tide of 2009 in Myrtle Beach, SC — in iCalendar format: BEGIN:VEVENT SUMMARY:Low Tide 0.39 ft DTSTART:20090101T090000Z UID:2009.0 DTSTAMP:20080527T000001Z END:VEVENT And here’s the equivalent XML: <vevent> <properties> <dtstamp> <date-time utc='yes'> <year>2008</year><month>5</month><day>27</day> <hour>0</hour><minute>0</minute><second>1</second> </date-time> </dtstamp> <dtstart> <date-time utc='yes'> <year>2009</year><month>1</month><day>1</day> <hour>9</hour><minute>0</minute><second>0</second> </date> </dtstart> <summary> <text>Low Tide 0.39 ft</text> </summary> <uid> <text>2009.0</text> </uid> </properties> </vevent> The mapping is quite straightforward, as you can see. At first glance, the XML version just seems verbose. So why bother? Because the iCalendar format can be tricky to read and write, either directly (using eyes and hands) or indirectly (using software). That’s especially true when, as is typical, events include longer chunks of text than you see here. I make an analogy to the RSS ecosystem. When I published my first RSS feed a decade ago, I wrote it by hand. More specifically, I copied an existing feed as a template, and altered it using cut-and-paste. Soon afterward, I wrote the first of countless scripts that flowed data through similar templates to produce various kinds of RSS feeds. Lots of other people did the same, and that’s part of the reason why we now have a robust network of RSS and Atom feeds that carries not only blogs, but all kinds of data packets. Another part of the reason is the Feed Validator which, thanks to heroic efforts by Mark Pilgrim and Sam Ruby, became and remains the essential sanity check for anybody who’s whipping up an ad-hoc RSS or Atom feed. No such ecosystem exists for iCalendar. I’ve been working hard to show why we need one, but the most compelling rationale comes from a Scott Adams essay that I quoted from in this blog entry. Dilber’s creator wrote: I think the biggest software revolution of the future is that the calendar will be the organizing filter for most of the information flowing into your life. You think you are bombarded with too much information every day, but in reality it is just the timing of the information that is wrong. Once the calendar becomes the organizing paradigm and filter, it won’t seem as if there is so much. If you buy that argument, then we’re going to need more than a handful of applications that can reliably create and exchange calendar data. We’ll want anyone to whip up a calendar feed as easily as anyone can now whip up an RSS/Atom feed. We’ll also need more than a handful of parsers that can reliably read calendar feeds, so that thousands of ad-hoc applications, services, and scripts will be able consume all the new streams of time-and-date-oriented information. I think that a standard XML representation of iCalendar will enable lots of ad-hoc producers and consumers to get into the game, and collectively bootstrap this new ecosystem. And that will enable what Scott Adams envisions. Here’s a small but evocative example. Yesterday I started up a new instance of the elmcity aggregator for Myrtle Beach, SC. The curator, Dave Slusher, found a tide table for his location, and it offers an iCalendar feed. So the Myrtle Beach calendar for today begins like this: Thu Jul 23 2009 WeeHours Thu 03:07 AM Low Tide -0.58 ft (Tide Table for Myrtle Beach, SC) Morning Thu 06:21 AM Sunrise 6:21 AM EDT (Tide Table for Myrtle Beach, SC) Thu 09:09 AM High Tide 5.99 ft (Tide Table for Myrtle Beach, SC) Thu 10:00 AM Free Coffee Fridays (eventful: ) Thu 10:00 AM Summer Arts Project at The Market Common (eventful: ) Thu 10:00 AM E.B. Lewis: Story Painter (eventful: ) Imagine this kind of thing happening on the scale of the RSS/Atom feed ecosystem. The lack of an agreed-upon XML representation for iCalendar isn’t the only reason why we don’t have an equally vibrant ecosystem of calendar feeds. But it’s an impediment that can be swept away, and I hope this proposal will finally do that. On this week’s Innovators show I got together with two of the authors of a new proposal for representing iCalendar in XML. Mike Douglass is lead developer of the Bedework Calendar System, and Steven Lees is Microsoft’s program manager for FeedSync and chair of the XML technical committee in CalConnect, the Calendaring and Scheduling Consortium. What’s proposed is no more, but no less, than a well-defined two-way mapping between the current non-XML-based iCalendar format and an equivalent XML format. So, for example, here’s an event — the first low tide of 2009 in Myrtle Beach, SC — in iCalendar format: BEGIN:VEVENT SUMMARY:Low Tide 0.39 ft DTSTART:20090101T090000Z UID:2009.0 DTSTAMP:20080527T000001Z END:VEVENT And here’s the equivalent XML: <vevent> <properties> <dtstamp> <date-time utc='yes'> <year>2008</year><month>5</month><day>27</day> <hour>0</hour><minute>0</minute><second>1</second> </date-time> </dtstamp> <dtstart> <date-time utc='yes'> <year>2009</year><month>1</month><day>1</day> <hour>9</hour><minute>0</minute><second>0</second> </date> </dtstart> <summary> <text>Low Tide 0.39 ft</text> </summary> <uid> <text>2009.0</text> </uid> </properties> </vevent> The mapping is quite straightforward, as you can see. At first glance, the XML version just seems verbose. So why bother? Because the iCalendar format can be tricky to read and write, either directly (using eyes and hands) or indirectly (using software). That’s especially true when, as is typical, events include longer chunks of text than you see here. I make an analogy to the RSS ecosystem. When I published my first RSS feed a decade ago, I wrote it by hand. More specifically, I copied an existing feed as a template, and altered it using cut-and-paste. Soon afterward, I wrote the first of countless scripts that flowed data through similar templates to produce various kinds of RSS feeds. Lots of other people did the same, and that’s part of the reason why we now have a robust network of RSS and Atom feeds that carries not only blogs, but all kinds of data packets. Another part of the reason is the Feed Validator which, thanks to heroic efforts by Mark Pilgrim and Sam Ruby, became and remains the essential sanity check for anybody who’s whipping up an ad-hoc RSS or Atom feed. No such ecosystem exists for iCalendar. I’ve been working hard to show why we need one, but the most compelling rationale comes from a Scott Adams essay that I quoted from in this blog entry. Dilber’s creator wrote: I think the biggest software revolution of the future is that the calendar will be the organizing filter for most of the information flowing into your life. You think you are bombarded with too much information every day, but in reality it is just the timing of the information that is wrong. Once the calendar becomes the organizing paradigm and filter, it won’t seem as if there is so much. If you buy that argument, then we’re going to need more than a handful of applications that can reliably create and exchange calendar data. We’ll want anyone to whip up a calendar feed as easily as anyone can now whip up an RSS/Atom feed. We’ll also need more than a handful of parsers that can reliably read calendar feeds, so that thousands of ad-hoc applications, services, and scripts will be able consume all the new streams of time-and-date-oriented information. I think that a standard XML representation of iCalendar will enable lots of ad-hoc producers and consumers to get into the game, and collectively bootstrap this new ecosystem. And that will enable what Scott Adams envisions. Here’s a small but evocative example. Yesterday I started up a new instance of the elmcity aggregator for Myrtle Beach, SC. The curator, Dave Slusher, found a tide table for his location, and it offers an iCalendar feed. So the Myrtle Beach calendar for today begins like this: Thu Jul 23 2009 WeeHours Thu 03:07 AM Low Tide -0.58 ft (Tide Table for Myrtle Beach, SC) Morning Thu 06:21 AM Sunrise 6:21 AM EDT (Tide Table for Myrtle Beach, SC) Thu 09:09 AM High Tide 5.99 ft (Tide Table for Myrtle Beach, SC) Thu 10:00 AM Free Coffee Fridays (eventful: ) Thu 10:00 AM Summer Arts Project at The Market Common (eventful: ) Thu 10:00 AM E.B. Lewis: Story Painter (eventful: ) Imagine this kind of thing happening on the scale of the RSS/Atom feed ecosystem. The lack of an agreed-upon XML representation for iCalendar isn’t the only reason why we don’t have an equally vibrant ecosystem of calendar feeds. But it’s an impediment that can be swept away, and I hope this proposal will finally do that. tag:odeo.com,2009-07-23,24851039 Thu, 23 Jul 2009 07:44:26 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24822327-Late-July-in-Toronto-DemoCamp-and-Science-2-0 On Tuesday July 28 I’ll be at the Toronto DemoCamp. I’m looking forward to meeting the designers, developers, and developers who’ll be there, seeing what you’re working on, and showing you what I’m working on. The following day I’ll be speaking at a Science 2.0 event organized by my friend Greg Wilson. Here are the forward-thinking scientists I’ll be joining: Titus Brown: Choosing Infrastructure and Testing Tools for Scientific Software Projects Cameron Neylon: A Web Native Research Record: Applying the Best of the Web to the Lab Notebook Michael Nielsen: Doing Science in the Open: How Online Tools are Changing Scientific Discovery David Rich: Using “Desktop” Languages for Big Problems Victoria Stodden: How Computational Science is Changing the Scientific Method I am not a scientest, nor do I play one on TV, so why me? Because back in 2000, Greg commissioned me to write a report entitled Internet Groupware for Scientific Collabor... On Tuesday July 28 I’ll be at the Toronto DemoCamp. I’m looking forward to meeting the designers, developers, and developers who’ll be there, seeing what you’re working on, and showing you what I’m working on. The following day I’ll be speaking at a Science 2.0 event organized by my friend Greg Wilson. Here are the forward-thinking scientists I’ll be joining: Titus Brown: Choosing Infrastructure and Testing Tools for Scientific Software Projects Cameron Neylon: A Web Native Research Record: Applying the Best of the Web to the Lab Notebook Michael Nielsen: Doing Science in the Open: How Online Tools are Changing Scientific Discovery David Rich: Using “Desktop” Languages for Big Problems Victoria Stodden: How Computational Science is Changing the Scientific Method I am not a scientest, nor do I play one on TV, so why me? Because back in 2000, Greg commissioned me to write a report entitled Internet Groupware for Scientific Collaboration. Greg was then working with the Los Alamos National Laboratory on ways to help scientists make better use of the tools of computation as well as the methods of online collaboration. I had recently finished my book Practical Internet Groupware, I was exploring what we would now call the Web 2.0 landscape, and I was thinking and writing a lot about how these open and loosely-coupled modes of communication could enable the sort of collaboration at the core of science (and other kinds of academic endeavors) in powerful new ways. Nearly a decade later, that vision is becoming a reality. I’m really excited to meet these folks, whose adventures I’ve been following through their blogs, and hear about their experiences at the forefront of what I believe will be a new golden age of science. In my own talk, I’ll review how own current project tackles the challenge of social information management, and aims to democratize the computational way of thinking that enables us to wire the web. On Tuesday July 28 I’ll be at the Toronto DemoCamp. I’m looking forward to meeting the designers, developers, and developers who’ll be there, seeing what you’re working on, and showing you what I’m working on. The following day I’ll be speaking at a Science 2.0 event organized by my friend Greg Wilson. Here are the forward-thinking scientists I’ll be joining: Titus Brown: Choosing Infrastructure and Testing Tools for Scientific Software Projects Cameron Neylon: A Web Native Research Record: Applying the Best of the Web to the Lab Notebook Michael Nielsen: Doing Science in the Open: How Online Tools are Changing Scientific Discovery David Rich: Using “Desktop” Languages for Big Problems Victoria Stodden: How Computational Science is Changing the Scientific Method I am not a scientest, nor do I play one on TV, so why me? Because back in 2000, Greg commissioned me to write a report entitled Internet Groupware for Scientific Collaboration. Greg was then working with the Los Alamos National Laboratory on ways to help scientists make better use of the tools of computation as well as the methods of online collaboration. I had recently finished my book Practical Internet Groupware, I was exploring what we would now call the Web 2.0 landscape, and I was thinking and writing a lot about how these open and loosely-coupled modes of communication could enable the sort of collaboration at the core of science (and other kinds of academic endeavors) in powerful new ways. Nearly a decade later, that vision is becoming a reality. I’m really excited to meet these folks, whose adventures I’ve been following through their blogs, and hear about their experiences at the forefront of what I believe will be a new golden age of science. In my own talk, I’ll review how own current project tackles the challenge of social information management, and aims to democratize the computational way of thinking that enables us to wire the web. tag:odeo.com,2009-07-17,24822327 Fri, 17 Jul 2009 11:06:27 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24816299-Tinker-to-Evers-to-Chance-Tripit-to-Dopplr-to-Facebook A few months back I observed: Tripit, meet Dopplr. Dopplr, Tripit. You two should really get to know one another. Richard Akerman replied: You can feed TripIt’s ical output into Dopplr, I hear (I haven’t tried it) That remark should have rung a loud bell for me, but somehow it didn’t. Then, yesterday, in conversation with James Senior, the bell rang. We were talking about how many services publish and/or subscribe to iCalendar feeds, how few people know that, and how much latent capability is being left on the table. Paraphrasing James: I’ll give you a perfect example. I use Tripit, it’s a wonderful service. You email it your travel itinerary, and it organizes all your information for you. But I’ve been frustrated not to be able to share that information with my friends on Facebook. I also use Dopplr, and Dopplr talks to Facebook, but Tripit doesn’t. Then I realized that Tripit publishes an iCalendar feed, and that Dopplr can subscribe to iC... A few months back I observed: Tripit, meet Dopplr. Dopplr, Tripit. You two should really get to know one another. Richard Akerman replied: You can feed TripIt’s ical output into Dopplr, I hear (I haven’t tried it) That remark should have rung a loud bell for me, but somehow it didn’t. Then, yesterday, in conversation with James Senior, the bell rang. We were talking about how many services publish and/or subscribe to iCalendar feeds, how few people know that, and how much latent capability is being left on the table. Paraphrasing James: I’ll give you a perfect example. I use Tripit, it’s a wonderful service. You email it your travel itinerary, and it organizes all your information for you. But I’ve been frustrated not to be able to share that information with my friends on Facebook. I also use Dopplr, and Dopplr talks to Facebook, but Tripit doesn’t. Then I realized that Tripit publishes an iCalendar feed, and that Dopplr can subscribe to iCalendar feeds. So I made that connection, and now my Tripit events are showing up in Facebook. Brilliant. Look: How did I miss that? Me, of all people, Mr. Splice-Everything-To-Everything, Mr. Find-Unintended-Uses-Of-Software, Mr. Cosmic-Significance-Of-Pub-Sub, Mr. Champion-Of-The-Underutilized-iCalendar-Standard, Mr. Computational-Thinking? Because wiring the web is still too abstract, too convoluted, and too non-obvious — even, sometimes, for me. The phrase wiring the web comes from Ray Ozzie, by the way. At ETech in 2006, demoed a concept called Live Clipboard. From my InfoWorld writeup: Subscribing to an RSS feed, for example, has never conformed to any familiar user-interface pattern. Soon copying and pasting RSS feeds will feel natural to everyone, and Ozzie hopes the copy/paste metaphor will also make advanced capabilities more accessible. Consider my LibraryLookup bookmarklet. Dragging it onto the browser’s toolbar isn’t something easily understood or explained. Using the clipboard as the wiring junction will make a lot more sense to most people. The same metaphor can accommodate what I’ve called lightweight service composition and what Ozzie calls “wiring the Web.” He showed how RSS feeds acting as service end points can be pasted into apps to create dynamically updating views. Virtually anyone can master this Tinkertoy approach to self-serve mashups. This was, and remains, a crucial insight. From now on, we are all going to be wiring the web in one way or another. And we’re going to need a conceptual frame in which to do that — ideally, a user-interface metaphor that’s already familiar. Maybe it’s as simple as copy/paste. Maybe it’s more like Yahoo! Pipes or Popfly blocks. Whatever it turns out to be, we need to invent and deploy a universal junction box for wiring the web. A few months back I observed: Tripit, meet Dopplr. Dopplr, Tripit. You two should really get to know one another. Richard Akerman replied: You can feed TripIt’s ical output into Dopplr, I hear (I haven’t tried it) That remark should have rung a loud bell for me, but somehow it didn’t. Then, yesterday, in conversation with James Senior, the bell rang. We were talking about how many services publish and/or subscribe to iCalendar feeds, how few people know that, and how much latent capability is being left on the table. Paraphrasing James: I’ll give you a perfect example. I use Tripit, it’s a wonderful service. You email it your travel itinerary, and it organizes all your information for you. But I’ve been frustrated not to be able to share that information with my friends on Facebook. I also use Dopplr, and Dopplr talks to Facebook, but Tripit doesn’t. Then I realized that Tripit publishes an iCalendar feed, and that Dopplr can subscribe to iCalendar feeds. So I made that connection, and now my Tripit events are showing up in Facebook. Brilliant. Look: How did I miss that? Me, of all people, Mr. Splice-Everything-To-Everything, Mr. Find-Unintended-Uses-Of-Software, Mr. Cosmic-Significance-Of-Pub-Sub, Mr. Champion-Of-The-Underutilized-iCalendar-Standard, Mr. Computational-Thinking? Because wiring the web is still too abstract, too convoluted, and too non-obvious — even, sometimes, for me. The phrase wiring the web comes from Ray Ozzie, by the way. At ETech in 2006, demoed a concept called Live Clipboard. From my InfoWorld writeup: Subscribing to an RSS feed, for example, has never conformed to any familiar user-interface pattern. Soon copying and pasting RSS feeds will feel natural to everyone, and Ozzie hopes the copy/paste metaphor will also make advanced capabilities more accessible. Consider my LibraryLookup bookmarklet. Dragging it onto the browser’s toolbar isn’t something easily understood or explained. Using the clipboard as the wiring junction will make a lot more sense to most people. The same metaphor can accommodate what I’ve called lightweight service composition and what Ozzie calls “wiring the Web.” He showed how RSS feeds acting as service end points can be pasted into apps to create dynamically updating views. Virtually anyone can master this Tinkertoy approach to self-serve mashups. This was, and remains, a crucial insight. From now on, we are all going to be wiring the web in one way or another. And we’re going to need a conceptual frame in which to do that — ideally, a user-interface metaphor that’s already familiar. Maybe it’s as simple as copy/paste. Maybe it’s more like Yahoo! Pipes or Popfly blocks. Whatever it turns out to be, we need to invent and deploy a universal junction box for wiring the web. tag:odeo.com,2009-07-16,24816299 Thu, 16 Jul 2009 08:39:00 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24808596-Talking-with-Peter-O%E2%80%99Toole-about-gathering-clinical-data-and-sharing-medical-knowledge My guest for this week’s Innovators show is Peter O’Toole from mTuitive, a company whose authoring toolkit for clinical data collection I featured in a 2006 screencast. mTuitive is working at the intersection of a number of disciplines that all need to come together to deliver cheaper and better health care. First, usability. Designing clinical data gathering systems that capture what’s right for the patient, along with what’s mandated by the insurance company, requires a careful balancing of constraints and freedom in software user interfaces. Second, knowledge engineering. Clinical systems don’t merely record data, they embody medical protocols that reflect an ever-changing consensus about methods and best practices. mTuitive’s authoring system aims to enable leading practioners to encode that knowledge in ways that can then guide others. But knowledge grows at the edge as well as at the center. So mTuitive also enables practitioners to extend a... My guest for this week’s Innovators show is Peter O’Toole from mTuitive, a company whose authoring toolkit for clinical data collection I featured in a 2006 screencast. mTuitive is working at the intersection of a number of disciplines that all need to come together to deliver cheaper and better health care. First, usability. Designing clinical data gathering systems that capture what’s right for the patient, along with what’s mandated by the insurance company, requires a careful balancing of constraints and freedom in software user interfaces. Second, knowledge engineering. Clinical systems don’t merely record data, they embody medical protocols that reflect an ever-changing consensus about methods and best practices. mTuitive’s authoring system aims to enable leading practioners to encode that knowledge in ways that can then guide others. But knowledge grows at the edge as well as at the center. So mTuitive also enables practitioners to extend and modify the software, injecting local knowledge and custom. Who owns this knowledge? Who’s liable for the consequences of its use? These are some of the implications we discussed. Third, semantics. Electronic medical records are still mainly narrative in form, says Peter O’Toole. But we’re moving toward more computable ways of describing observations about, say, the nature and size of tumors. Fourth, social software. My hunch, and Peter O’Toole’s too, is that progress toward the nirvana of medical records that are both semantically rich and interoperable will be powered by a two-stroke engine. One stroke of the piston will be driven by centrally-defined standards and centrally-imposed legislation. But the other will be driven by networked collaboration, at the edge, among doctors who pool and codify their experiential knowledge using ad-hoc, Web 2.0-like methods. My guest for this week’s Innovators show is Peter O’Toole from mTuitive, a company whose authoring toolkit for clinical data collection I featured in a 2006 screencast. mTuitive is working at the intersection of a number of disciplines that all need to come together to deliver cheaper and better health care. First, usability. Designing clinical data gathering systems that capture what’s right for the patient, along with what’s mandated by the insurance company, requires a careful balancing of constraints and freedom in software user interfaces. Second, knowledge engineering. Clinical systems don’t merely record data, they embody medical protocols that reflect an ever-changing consensus about methods and best practices. mTuitive’s authoring system aims to enable leading practioners to encode that knowledge in ways that can then guide others. But knowledge grows at the edge as well as at the center. So mTuitive also enables practitioners to extend and modify the software, injecting local knowledge and custom. Who owns this knowledge? Who’s liable for the consequences of its use? These are some of the implications we discussed. Third, semantics. Electronic medical records are still mainly narrative in form, says Peter O’Toole. But we’re moving toward more computable ways of describing observations about, say, the nature and size of tumors. Fourth, social software. My hunch, and Peter O’Toole’s too, is that progress toward the nirvana of medical records that are both semantically rich and interoperable will be powered by a two-stroke engine. One stroke of the piston will be driven by centrally-defined standards and centrally-imposed legislation. But the other will be driven by networked collaboration, at the edge, among doctors who pool and codify their experiential knowledge using ad-hoc, Web 2.0-like methods. tag:odeo.com,2009-07-15,24808596 Wed, 15 Jul 2009 07:14:31 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24801472-Hat-tip-to-Joshua-Allen%E2%80%99s-Better-Living-Through-Software Here’s another piece of Say Everything that I want to spotlight: Microsoft wasn’t known as a haven of openness and cooperation. But it was a big place with a lot of smart people. At the turn of the millenium, during the company’s bitter antitrust fight with the U.S. Department of Justice, many of those people found it impossible to recognize themselves in the press’s portrait of the company. The first programmer at Microsoft to start blogging, Joshua Allen, set himself up with an account on Dave Winer’s EditThisPage service in 2000 and started posting under the header “Better Living Through Software: Tales of Life at Microsoft.” It was totally informal and unauthorized — a lone call for a parley raised from behind the company’s siege walls. Allen explained his intent: “I wanted to say that I am a Microsoft person and you can talk with me.” I used to read Joshua’s blog back then, I still read it now, it was nice ... Here’s another piece of Say Everything that I want to spotlight: Microsoft wasn’t known as a haven of openness and cooperation. But it was a big place with a lot of smart people. At the turn of the millenium, during the company’s bitter antitrust fight with the U.S. Department of Justice, many of those people found it impossible to recognize themselves in the press’s portrait of the company. The first programmer at Microsoft to start blogging, Joshua Allen, set himself up with an account on Dave Winer’s EditThisPage service in 2000 and started posting under the header “Better Living Through Software: Tales of Life at Microsoft.” It was totally informal and unauthorized — a lone call for a parley raised from behind the company’s siege walls. Allen explained his intent: “I wanted to say that I am a Microsoft person and you can talk with me.” I used to read Joshua’s blog back then, I still read it now, it was nice to see its seminal role acknowledged in the book. Here’s a picture of the blog’s home page, annotated by the ClearForest Gnosis entity extractor: Quite a cast of characters! Here’s another piece of Say Everything that I want to spotlight: Microsoft wasn’t known as a haven of openness and cooperation. But it was a big place with a lot of smart people. At the turn of the millenium, during the company’s bitter antitrust fight with the U.S. Department of Justice, many of those people found it impossible to recognize themselves in the press’s portrait of the company. The first programmer at Microsoft to start blogging, Joshua Allen, set himself up with an account on Dave Winer’s EditThisPage service in 2000 and started posting under the header “Better Living Through Software: Tales of Life at Microsoft.” It was totally informal and unauthorized — a lone call for a parley raised from behind the company’s siege walls. Allen explained his intent: “I wanted to say that I am a Microsoft person and you can talk with me.” I used to read Joshua’s blog back then, I still read it now, it was nice to see its seminal role acknowledged in the book. Here’s a picture of the blog’s home page, annotated by the ClearForest Gnosis entity extractor: Quite a cast of characters! tag:odeo.com,2009-07-14,24801472 Tue, 14 Jul 2009 13:45:06 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24801473-More-fun-than-herding-servers Until recently, the elmcity calendar aggregator was running as a single instance of an Azure worker role. The idea all along, of course, was to exploit the system’s ability to farm out the work of aggregation to many workers. Although the sixteen cities currently being aggregated don’t yet require the service to scale beyond a single instance, I’d been meaning to lay the foundation for that. This week I finally did. Will there ever be hundreds or thousands of participating cities and towns? Maybe that’ll happen, maybe it won’t, but the gating factor will not be my ability to babysit servers. That’s a remarkable change from just a few years ago. Over the weekend I read Scott Rosenberg’s new history of blogging, Say Everything. Here’s a poignant moment from 2001: Blogger still lived a touch-and-go existence. Its expenses had dropped from a $50,000-a-month burn rate to a few thousand in rent and technical costs for bandwidth and such; sti... Until recently, the elmcity calendar aggregator was running as a single instance of an Azure worker role. The idea all along, of course, was to exploit the system’s ability to farm out the work of aggregation to many workers. Although the sixteen cities currently being aggregated don’t yet require the service to scale beyond a single instance, I’d been meaning to lay the foundation for that. This week I finally did. Will there ever be hundreds or thousands of participating cities and towns? Maybe that’ll happen, maybe it won’t, but the gating factor will not be my ability to babysit servers. That’s a remarkable change from just a few years ago. Over the weekend I read Scott Rosenberg’s new history of blogging, Say Everything. Here’s a poignant moment from 2001: Blogger still lived a touch-and-go existence. Its expenses had dropped from a $50,000-a-month burn rate to a few thousand in rent and technical costs for bandwidth and such; still, even that modest budget wasn’t easy to meet. Eventually [Evan] Williams had to shut down the office entirely and move the servers into his apartment. He remembers this period as an emotional rollercoaster. “I don’t know how I’m going to pay the rent, and I can’t figure that out because the server’s not running, and I have to stay up all night, trying to figure out Linux, and being hacked, and then fix that.” I’ve been one of those guys who babysits the server under the desk, and I’m glad I won’t ever have to go back there again. What I will have to do, instead, is learn how to take advantage of the cloud resources now becoming available. But I’m finding that to be an enjoyable challenge. In the case of the calendar aggregator, which needs to map many worker roles to many cities, I’m using a blackboard approach. Here’s a snapshot of it, from an aggregator run using only a single worker instance: id: westlafcals start: 7/14/2009 12:12:05 PM stop: 7/14/2009 12:14:46 PM running: False id: networksierra start: 7/14/2009 12:14:48 PM stop: 7/14/2009 12:15:05 PM running: False id: localist start: 7/14/2009 12:15:06 PM stop: 7/14/2009 5:37:03 AM running: True id: aroundfred start: 7/14/2009 5:37:05 AM stop: 7/14/2009 5:39:20 AM running: False The moving finger wrote westlafcals (West Lafayette) and networksierra (Sonora), it’s now writing localist (Baltimore), and will next write aroundfred (Fredericksburg). Here’s a snapshot from another run using two worker instances: id: westlafcals start: 7/14/2009 10:12:05 PM stop: 7/14/2009 4:37:03 AM running: True id: networksierra start: 7/14/2009 10:12:10 PM stop: 7/14/2009 10:13:05 PM running: False id: localist start: 7/14/2009 10:13:06 PM stop: 7/14/2009 4:41:12 AM running: True id: aroundfred start: 7/14/2009 4:41:05 AM stop: 7/14/2009 4:42:20 AM running: False Now there are two moving fingers. One’s writing westlafcals, one has written networksierra, one’s writing localist, and one or the other will soon write aroundfred. The total elapsed time will be very close to half what it was in the single-instance case. I’d love to crank up the instance count and see an aggregation run rip through all the cities in no time flat. But the Azure beta caps the instance count at two. The blackboard is an Azure table with one record for each city. Records are flexible bags of name/value pairs. If you make a REST call to the table service to query for one of those records, the Atom payload that comes back looks like this: <m:properties> <d:PartitionKey>blackboard</d:PartitionKey> <d:RowKey>aroundfred</d:RowKey> <d:start>7/14/2009 4:41:05 AM</d:start> <d:stop>7/14/2009 4:42:20 AM</d:stop> <d:running>False</d:stop> </m:properties> At the start of a cycle, each worker wakes up, iterates through all the cities, aggregates those not claimed by other workers, and then sleeps until the next cycle. To claim a city, a worker tries to create a record in a parallel Azure table, using the PartitionKey locks instead of blackboard. If the worker succeeds in doing that, it considers the city locked for its own use, it aggregates the city’s calendars, and then it deletes the lock record. If the worker fails to create that record, it considers the city locked by another worker and moves on. This cycle is currently one hour. But in order to respect the various services it pulls from, the service defines the interval between aggregation runs to be 8 hours. So when a worker claims a city, it first checks to see if the last aggregation started more than 8 hours ago. If not, the worker skips that city. Locks can be abandoned. That could happen if a worker hangs or crashes, or when I redeploy a new version of the service. So the worker also checks to see if a lock has been hanging around longer than the aggregation interval. If so, it overrides the lock and aggregates that city. I’m sure this scheme isn’t bulletproof, but I reckon it doesn’t need to be. If two workers should happen to wind up aggregating the same city at about the same time, it’s no big deal. The last writer wins, a little extra work gets done. Anyway, I’ll be watching the blackboard over the next few days. There’s undoubtedly more tinkering to do. And it’s a lot more fun than herding servers. Until recently, the elmcity calendar aggregator was running as a single instance of an Azure worker role. The idea all along, of course, was to exploit the system’s ability to farm out the work of aggregation to many workers. Although the sixteen cities currently being aggregated don’t yet require the service to scale beyond a single instance, I’d been meaning to lay the foundation for that. This week I finally did. Will there ever be hundreds or thousands of participating cities and towns? Maybe that’ll happen, maybe it won’t, but the gating factor will not be my ability to babysit servers. That’s a remarkable change from just a few years ago. Over the weekend I read Scott Rosenberg’s new history of blogging, Say Everything. Here’s a poignant moment from 2001: Blogger still lived a touch-and-go existence. Its expenses had dropped from a $50,000-a-month burn rate to a few thousand in rent and technical costs for bandwidth and such; still, even that modest budget wasn’t easy to meet. Eventually [Evan] Williams had to shut down the office entirely and move the servers into his apartment. He remembers this period as an emotional rollercoaster. “I don’t know how I’m going to pay the rent, and I can’t figure that out because the server’s not running, and I have to stay up all night, trying to figure out Linux, and being hacked, and then fix that.” I’ve been one of those guys who babysits the server under the desk, and I’m glad I won’t ever have to go back there again. What I will have to do, instead, is learn how to take advantage of the cloud resources now becoming available. But I’m finding that to be an enjoyable challenge. In the case of the calendar aggregator, which needs to map many worker roles to many cities, I’m using a blackboard approach. Here’s a snapshot of it, from an aggregator run using only a single worker instance: id: westlafcals start: 7/14/2009 12:12:05 PM stop: 7/14/2009 12:14:46 PM running: False id: networksierra start: 7/14/2009 12:14:48 PM stop: 7/14/2009 12:15:05 PM running: False id: localist start: 7/14/2009 12:15:06 PM stop: 7/14/2009 5:37:03 AM running: True id: aroundfred start: 7/14/2009 5:37:05 AM stop: 7/14/2009 5:39:20 AM running: False The moving finger wrote westlafcals (West Lafayette) and networksierra (Sonora), it’s now writing localist (Baltimore), and will next write aroundfred (Fredericksburg). Here’s a snapshot from another run using two worker instances: id: westlafcals start: 7/14/2009 10:12:05 PM stop: 7/14/2009 4:37:03 AM running: True id: networksierra start: 7/14/2009 10:12:10 PM stop: 7/14/2009 10:13:05 PM running: False id: localist start: 7/14/2009 10:13:06 PM stop: 7/14/2009 4:41:12 AM running: True id: aroundfred start: 7/14/2009 4:41:05 AM stop: 7/14/2009 4:42:20 AM running: False Now there are two moving fingers. One’s writing westlafcals, one has written networksierra, one’s writing localist, and one or the other will soon write aroundfred. The total elapsed time will be very close to half what it was in the single-instance case. I’d love to crank up the instance count and see an aggregation run rip through all the cities in no time flat. But the Azure beta caps the instance count at two. The blackboard is an Azure table with one record for each city. Records are flexible bags of name/value pairs. If you make a REST call to the table service to query for one of those records, the Atom payload that comes back looks like this: <m:properties> <d:PartitionKey>blackboard</d:PartitionKey> <d:RowKey>aroundfred</d:RowKey> <d:start>7/14/2009 4:41:05 AM</d:start> <d:stop>7/14/2009 4:42:20 AM</d:stop> <d:running>False</d:stop> </m:properties> At the start of a cycle, each worker wakes up, iterates through all the cities, aggregates those not claimed by other workers, and then sleeps until the next cycle. To claim a city, a worker tries to create a record in a parallel Azure table, using the PartitionKey locks instead of blackboard. If the worker succeeds in doing that, it considers the city locked for its own use, it aggregates the city’s calendars, and then it deletes the lock record. If the worker fails to create that record, it considers the city locked by another worker and moves on. This cycle is currently one hour. But in order to respect the various services it pulls from, the service defines the interval between aggregation runs to be 8 hours. So when a worker claims a city, it first checks to see if the last aggregation started more than 8 hours ago. If not, the worker skips that city. Locks can be abandoned. That could happen if a worker hangs or crashes, or when I redeploy a new version of the service. So the worker also checks to see if a lock has been hanging around longer than the aggregation interval. If so, it overrides the lock and aggregates that city. I’m sure this scheme isn’t bulletproof, but I reckon it doesn’t need to be. If two workers should happen to wind up aggregating the same city at about the same time, it’s no big deal. The last writer wins, a little extra work gets done. Anyway, I’ll be watching the blackboard over the next few days. There’s undoubtedly more tinkering to do. And it’s a lot more fun than herding servers. tag:odeo.com,2009-07-14,24801473 Tue, 14 Jul 2009 13:11:50 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24793093-The-civic-dashboard On Friday my local paper ran a story entitled Keene crime rates steady over years. Because that link will go dark soon, I’m going to assert fair use for the part of the story that cites statistics: Strings of vehicle break-ins and vandalism and the occasional vicious beating or stabbing may lead some to believe that Keene’s streets are getting meaner, but crime statistics show little change over the last six years. Even in light of rough economic times, which typically parallel a spike in shoplifting — people begin stealing groceries or other necessities they can no longer afford — the Elm City’s property crime rate remains stable. The city’s social programs, such as The Community Kitchen, which provides food to area residents in need, play a significant role in curbing crime, Keene police Lt. Jay U. Duguay said. “We’re behind the nation when it comes to economic issues. People are still losing their homes and jobs, but overall we haven’t felt the effects of it yet,” he said. ... On Friday my local paper ran a story entitled Keene crime rates steady over years. Because that link will go dark soon, I’m going to assert fair use for the part of the story that cites statistics: Strings of vehicle break-ins and vandalism and the occasional vicious beating or stabbing may lead some to believe that Keene’s streets are getting meaner, but crime statistics show little change over the last six years. Even in light of rough economic times, which typically parallel a spike in shoplifting — people begin stealing groceries or other necessities they can no longer afford — the Elm City’s property crime rate remains stable. The city’s social programs, such as The Community Kitchen, which provides food to area residents in need, play a significant role in curbing crime, Keene police Lt. Jay U. Duguay said. “We’re behind the nation when it comes to economic issues. People are still losing their homes and jobs, but overall we haven’t felt the effects of it yet,” he said. “Right now it’s wait-and-see.” During the last six years, Keene police have received an average of 490 reports dealing with larceny or theft. Last year they took 667 reports of larceny or theft, the highest number of those types of crimes since 2002, which saw 604 reports. From the beginning of this year to the end of April, there were 202 reports of larceny and theft, slightly higher than the 147 during the same period last year, and 33 burglaries, which is on par with previous years. “There’s going to be periods with a little influx, but for the most part it’s steady,” Duguay said. “I was actually kind of surprised at how consistent the numbers were.” In 2004 and 2005, property crime rates dipped dramatically. While 2003 saw 557 larcenies and thefts, that number hit 272 the following year and then slightly increased to 286 the next year before rising to 455 in 2006. “We didn’t change our patrol procedures during those times (2004 and 2005) and we weren’t up to full staff. So I don’t know why those years are lower,” Duguay said. “I think the more consistent number is the high number, but thank goodness for the lows.” Violent crime reports in Keene have also remained steady over the last several years, with an average of 366 assaults annually. Between 20 and 30 sex assaults are reported in the city each year, though only a small fraction of those cases result in arrests because the others lack sufficient evidence, Duguay said. Statistics only tell part of the story, though. For the crime victims, the numbers hold little meaning. The story concludes with anecdotes from townsfolk who either do, or don’t, believe that tough economic times are making Keene’s streets meaner. I quote at length from it here because I think it captures a moment in time. The story seems to be data-driven, but not in the way that many of us now realize such stories can be. The reporter got some numbers from the police department, and the story quotes a lieutenant’s interpretation of those numbers, but there’s nothing available for an interested citizen to verify or falsify. And there’s no reference to an alternative source — from the US Bureau of Justice — that could confirm, challenge, or otherwise contextualize the numbers. I hope that my response, below, also marks a moment in time — one in which people didn’t demand, governments didn’t provide, journalists didn’t exploit, and all these groups didn’t collaboratively engage with more and better evidence than informs most civic dialogue today. From time to time, communities ask: Are we having a crime wave? A couple of summers ago it seemed that way. The Sentinel invited TalkBalk comments, and one person wrote: “Keene has gone downhill. Once a peaceful, quaint city that was safe, it is no more.” We shouldn’t have to just speculate about these trends though. We should be able to look at the data and draw reasonable conclusions. Increasingly, we can. In 2007 I looked, and the first source I found was the data reported by the Keene police department (and every other police department around the country) to the US Bureau of Justice. I noticed a couple of things. First, the numbers showed no uptick in violent crime. But since they stopped in 2005, they didn’t address concern about events in the then-current 2006-2007 period. Second, because the numbers went back to 1985, they revealed a remarkable anomaly. There was a huge spike in violent crime — assaults and rapes — from 1990 to 1994. You can see the trend plainly in the charts and data I’ve posted at http://jonudell.net/crime/keene-crime.html. What happened then? How should this historical context influence our perception of current trends? I’d love to see the Sentinel ask, and try to answer, these questions. Since the Bureau of Justice data wasn’t current enough to address the 2006-2007 concerns about crime, I asked the police department to provide me with more recent data. In the end, after multiple requests and some nudging by an attorney, they complied. The snapshot I received, with numbers through July 2007, showed no evidence of a recent uptick in either violent crime or property crime. That was reassuring. It was also enlightening to compare the raw data in the police spreadsheet to the numbers reported to the Bureau of Justice. They don’t exactly line up. This isn’t nefarious, it’s just what happens when local systems try to mesh with national systems. There is a lot of local variation in the classification of different types of crimes, and room for interpretation when you bundle them into larger categories. Fast forward to summer 2009. The economy has tanked, and people are again wondering whether we’re having a crime wave. The Sentinel gathered some data, talked to the police, and concluded — I suspect correctly — that as before, the perception of a crime wave is not the reality. For the reasons I’ve explained, the police department numbers reported in the Sentinel don’t quite line up with those reported to the Bureau of Justice. Consider larceny-theft, for example: 2003 2004 2005 2006 Sentinel 557 272 286 455 Justice 534 245 235 622 But I do wonder about this: “Violent crime reports in Keene have also remained steady over the last several years, with an average of 366 assaults annually.” I hope that’s an error. According to the Bureau of Justice there were at most about 100 violent crimes per year, back in the dark ages of 1990-1994, and we’ve averaged between 40 and 60 per year from then until 2007. In any case, here’s the larger point. Cities around the country have begun to realize that the operational data of city government can be made available to everyone — citizens as well as journalists — so that we can all monitor the health of our cities in a collaborative way. Crime statistics are one popular category of data, others include restaurant inspections, infrastructure repairs, and licensing. Nowadays it costs about $100/month to augment a police department’s information system with software that reports current crime statistics online, and also displays the locations of crimes on a map. In New Hampshire, one such system (crimereports.com) has been installed in Exeter, Hampton, Laconia, and Rochester. I’d love to see the Keene police department join that club. A civic dashboard is part of what I proposed during the Community Visioning Process. But there’s no need to wait until 2028. Cities around the country are creating their dashboards now, and we can too. On Friday my local paper ran a story entitled Keene crime rates steady over years. Because that link will go dark soon, I’m going to assert fair use for the part of the story that cites statistics: Strings of vehicle break-ins and vandalism and the occasional vicious beating or stabbing may lead some to believe that Keene’s streets are getting meaner, but crime statistics show little change over the last six years. Even in light of rough economic times, which typically parallel a spike in shoplifting — people begin stealing groceries or other necessities they can no longer afford — the Elm City’s property crime rate remains stable. The city’s social programs, such as The Community Kitchen, which provides food to area residents in need, play a significant role in curbing crime, Keene police Lt. Jay U. Duguay said. “We’re behind the nation when it comes to economic issues. People are still losing their homes and jobs, but overall we haven’t felt the effects of it yet,” he said. “Right now it’s wait-and-see.” During the last six years, Keene police have received an average of 490 reports dealing with larceny or theft. Last year they took 667 reports of larceny or theft, the highest number of those types of crimes since 2002, which saw 604 reports. From the beginning of this year to the end of April, there were 202 reports of larceny and theft, slightly higher than the 147 during the same period last year, and 33 burglaries, which is on par with previous years. “There’s going to be periods with a little influx, but for the most part it’s steady,” Duguay said. “I was actually kind of surprised at how consistent the numbers were.” In 2004 and 2005, property crime rates dipped dramatically. While 2003 saw 557 larcenies and thefts, that number hit 272 the following year and then slightly increased to 286 the next year before rising to 455 in 2006. “We didn’t change our patrol procedures during those times (2004 and 2005) and we weren’t up to full staff. So I don’t know why those years are lower,” Duguay said. “I think the more consistent number is the high number, but thank goodness for the lows.” Violent crime reports in Keene have also remained steady over the last several years, with an average of 366 assaults annually. Between 20 and 30 sex assaults are reported in the city each year, though only a small fraction of those cases result in arrests because the others lack sufficient evidence, Duguay said. Statistics only tell part of the story, though. For the crime victims, the numbers hold little meaning. The story concludes with anecdotes from townsfolk who either do, or don’t, believe that tough economic times are making Keene’s streets meaner. I quote at length from it here because I think it captures a moment in time. The story seems to be data-driven, but not in the way that many of us now realize such stories can be. The reporter got some numbers from the police department, and the story quotes a lieutenant’s interpretation of those numbers, but there’s nothing available for an interested citizen to verify or falsify. And there’s no reference to an alternative source — from the US Bureau of Justice — that could confirm, challenge, or otherwise contextualize the numbers. I hope that my response, below, also marks a moment in time — one in which people didn’t demand, governments didn’t provide, journalists didn’t exploit, and all these groups didn’t collaboratively engage with more and better evidence than informs most civic dialogue today. From time to time, communities ask: Are we having a crime wave? A couple of summers ago it seemed that way. The Sentinel invited TalkBalk comments, and one person wrote: “Keene has gone downhill. Once a peaceful, quaint city that was safe, it is no more.” We shouldn’t have to just speculate about these trends though. We should be able to look at the data and draw reasonable conclusions. Increasingly, we can. In 2007 I looked, and the first source I found was the data reported by the Keene police department (and every other police department around the country) to the US Bureau of Justice. I noticed a couple of things. First, the numbers showed no uptick in violent crime. But since they stopped in 2005, they didn’t address concern about events in the then-current 2006-2007 period. Second, because the numbers went back to 1985, they revealed a remarkable anomaly. There was a huge spike in violent crime — assaults and rapes — from 1990 to 1994. You can see the trend plainly in the charts and data I’ve posted at http://jonudell.net/crime/keene-crime.html. What happened then? How should this historical context influence our perception of current trends? I’d love to see the Sentinel ask, and try to answer, these questions. Since the Bureau of Justice data wasn’t current enough to address the 2006-2007 concerns about crime, I asked the police department to provide me with more recent data. In the end, after multiple requests and some nudging by an attorney, they complied. The snapshot I received, with numbers through July 2007, showed no evidence of a recent uptick in either violent crime or property crime. That was reassuring. It was also enlightening to compare the raw data in the police spreadsheet to the numbers reported to the Bureau of Justice. They don’t exactly line up. This isn’t nefarious, it’s just what happens when local systems try to mesh with national systems. There is a lot of local variation in the classification of different types of crimes, and room for interpretation when you bundle them into larger categories. Fast forward to summer 2009. The economy has tanked, and people are again wondering whether we’re having a crime wave. The Sentinel gathered some data, talked to the police, and concluded — I suspect correctly — that as before, the perception of a crime wave is not the reality. For the reasons I’ve explained, the police department numbers reported in the Sentinel don’t quite line up with those reported to the Bureau of Justice. Consider larceny-theft, for example: 2003 2004 2005 2006 Sentinel 557 272 286 455 Justice 534 245 235 622 But I do wonder about this: “Violent crime reports in Keene have also remained steady over the last several years, with an average of 366 assaults annually.” I hope that’s an error. According to the Bureau of Justice there were at most about 100 violent crimes per year, back in the dark ages of 1990-1994, and we’ve averaged between 40 and 60 per year from then until 2007. In any case, here’s the larger point. Cities around the country have begun to realize that the operational data of city government can be made available to everyone — citizens as well as journalists — so that we can all monitor the health of our cities in a collaborative way. Crime statistics are one popular category of data, others include restaurant inspections, infrastructure repairs, and licensing. Nowadays it costs about $100/month to augment a police department’s information system with software that reports current crime statistics online, and also displays the locations of crimes on a map. In New Hampshire, one such system (crimereports.com) has been installed in Exeter, Hampton, Laconia, and Rochester. I’d love to see the Keene police department join that club. A civic dashboard is part of what I proposed during the Community Visioning Process. But there’s no need to wait until 2028. Cities around the country are creating their dashboards now, and we can too. tag:odeo.com,2009-07-12,24793093 Sun, 12 Jul 2009 16:34:53 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24773599-Understanding-Wikipedia-notability Some fellow residents of my town have recently noticed, and pointed out to me, that I’m listed in Wikipedia as a notable inhabitant of Keene, NH. They’re more impressed than they should be. All forms of notability are subject to bias, but Internet notability is subject to a different kind of bias than most people realize. For example, friends and family used to be impressed by the fact that I was the top result in Google for my first name — and then second to Jon Stewart for a long while, until I had to reboot my InfoWorld archive. Why? Just because I’ve projected a large surface area of searchable documents whose titles include the trigram jon. An example of a far more notable person than me is Glenn Fine, who was in my grade in junior high school and is now Inspector General for the Department of Justice. You won’t find him anywhere near the top of a search for his first name because Inspectors General don’t (yet) project a large surface area of... Some fellow residents of my town have recently noticed, and pointed out to me, that I’m listed in Wikipedia as a notable inhabitant of Keene, NH. They’re more impressed than they should be. All forms of notability are subject to bias, but Internet notability is subject to a different kind of bias than most people realize. For example, friends and family used to be impressed by the fact that I was the top result in Google for my first name — and then second to Jon Stewart for a long while, until I had to reboot my InfoWorld archive. Why? Just because I’ve projected a large surface area of searchable documents whose titles include the trigram jon. An example of a far more notable person than me is Glenn Fine, who was in my grade in junior high school and is now Inspector General for the Department of Justice. You won’t find him anywhere near the top of a search for his first name because Inspectors General don’t (yet) project a large surface area of documents onto the web. To place my newfound Wikipedia notability into a similar context, I wanted to show people how these lists of notable inhabitants are made. I figured the person who made the change is somebody who knows of my work, because I’ve written about it so much online, and who is inclined to edit Wikipedia, which correlates with an interest in my work. I wanted to illustrate exactly who, when, and how, so I went to Wikipedia with the confident expectation that it would be easy to answer those questions. Surprisingly, it wasn’t. I guess I haven’t really tried searching revision histories in Wikipedia before, but in this case and a few others I’ve tried lately, it seems quite difficult to pinpoint the author of a change. For example, on Twitter I asked: Wikipedia: “The term ‘Web 2.0′ was coined by Darcy DiNucci in 1999.” Added when, by whom? WikiBlame seems an ineffective way to find out. @bazzargh replied: Robert Gehl. http://bit.ly/46r1a Thanks. By the way, how’d you do that? switch to 500 view in history, then rough bisection from oldest. Couple of minutes; used this a lot to find long-lived vandalism. if older, I progressively back off 2..4..8… pages through this. In this case though, there was a clueful log message! That’s pretty much what I’ve found myself doing when trying to track down changes, so I was glad to know it wasn’t just me. But this highlights an important point about transparency: It’s all relative. One of the reasons we think of government as opaque is that while records may be notionally public, it takes time, effort, and skill to visit city hall, dig through them, and find what you’re looking for. I have always regarded Wikipedia as an extreme counter-example. And that’s true. It is radically transparent. You can ultimately find out exactly how any statement in any article came to be. You may not be able to correlate the author’s pseudonym to a real-world identity, but you can evaluate that author’s corpus and reputation within the context of Wikipedia. And yet, the ability to do this spelunking requires more time, effort, and skill than most people possess. Although I’m reluctant to deflate my status as a notable inhabitant of Keene, I wish it were easier for people who read that to also find out what it does — and doesn’t — mean. Some fellow residents of my town have recently noticed, and pointed out to me, that I’m listed in Wikipedia as a notable inhabitant of Keene, NH. They’re more impressed than they should be. All forms of notability are subject to bias, but Internet notability is subject to a different kind of bias than most people realize. For example, friends and family used to be impressed by the fact that I was the top result in Google for my first name — and then second to Jon Stewart for a long while, until I had to reboot my InfoWorld archive. Why? Just because I’ve projected a large surface area of searchable documents whose titles include the trigram jon. An example of a far more notable person than me is Glenn Fine, who was in my grade in junior high school and is now Inspector General for the Department of Justice. You won’t find him anywhere near the top of a search for his first name because Inspectors General don’t (yet) project a large surface area of documents onto the web. To place my newfound Wikipedia notability into a similar context, I wanted to show people how these lists of notable inhabitants are made. I figured the person who made the change is somebody who knows of my work, because I’ve written about it so much online, and who is inclined to edit Wikipedia, which correlates with an interest in my work. I wanted to illustrate exactly who, when, and how, so I went to Wikipedia with the confident expectation that it would be easy to answer those questions. Surprisingly, it wasn’t. I guess I haven’t really tried searching revision histories in Wikipedia before, but in this case and a few others I’ve tried lately, it seems quite difficult to pinpoint the author of a change. For example, on Twitter I asked: Wikipedia: “The term ‘Web 2.0′ was coined by Darcy DiNucci in 1999.” Added when, by whom? WikiBlame seems an ineffective way to find out. @bazzargh replied: Robert Gehl. http://bit.ly/46r1a Thanks. By the way, how’d you do that? switch to 500 view in history, then rough bisection from oldest. Couple of minutes; used this a lot to find long-lived vandalism. if older, I progressively back off 2..4..8… pages through this. In this case though, there was a clueful log message! That’s pretty much what I’ve found myself doing when trying to track down changes, so I was glad to know it wasn’t just me. But this highlights an important point about transparency: It’s all relative. One of the reasons we think of government as opaque is that while records may be notionally public, it takes time, effort, and skill to visit city hall, dig through them, and find what you’re looking for. I have always regarded Wikipedia as an extreme counter-example. And that’s true. It is radically transparent. You can ultimately find out exactly how any statement in any article came to be. You may not be able to correlate the author’s pseudonym to a real-world identity, but you can evaluate that author’s corpus and reputation within the context of Wikipedia. And yet, the ability to do this spelunking requires more time, effort, and skill than most people possess. Although I’m reluctant to deflate my status as a notable inhabitant of Keene, I wish it were easier for people who read that to also find out what it does — and doesn’t — mean. tag:odeo.com,2009-07-09,24773599 Thu, 09 Jul 2009 05:31:59 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24764784-Strategic-choices-for-calendar-publishers Although I haven’t been able to confirm this officially yet, it looks like FuseCal, the HTML screen-scraping service that I’ve been using (and recommending) as a way to convert calendar-like web pages into iCalendar feeds, has shut down. The web pages that FuseCal has been successfully processing, for several curators participating in the elmcity project, are listed below. They’re a kind of existence proof, validating the notion that unstructured calendar info — what people intuitively create — can be mechanically transformed into structured info that syndicates reliably. I hope this service, or some future variant of it, will continue. It’s a really useful way to help people grasp the concept of publishing calendar feeds. But in the long run, it’s a set of training wheels. Ultimately we need to teach people why and how to produce such feeds more directly. All of the event information shown below could be managed in a more structured way usi... Although I haven’t been able to confirm this officially yet, it looks like FuseCal, the HTML screen-scraping service that I’ve been using (and recommending) as a way to convert calendar-like web pages into iCalendar feeds, has shut down. The web pages that FuseCal has been successfully processing, for several curators participating in the elmcity project, are listed below. They’re a kind of existence proof, validating the notion that unstructured calendar info — what people intuitively create — can be mechanically transformed into structured info that syndicates reliably. I hope this service, or some future variant of it, will continue. It’s a really useful way to help people grasp the concept of publishing calendar feeds. But in the long run, it’s a set of training wheels. Ultimately we need to teach people why and how to produce such feeds more directly. All of the event information shown below could be managed in a more structured way using calendar software that produces data feeds for syndication and web pages for browsing. More broadly, incidents like this prompt us to consider the nature of the services ecosystem we’re all embedded in — as users and, increasingly, as co-creators. In the software business, developers have long since learned to evaluate the benefits and risks of “taking a dependency” on a component, library, or service. Users didn’t have to think too much about that. A software product that was discontinued would keep working perfectly well, maybe for years. But services can — and sometimes do — stop abruptly. Since the elmcity project is embedded in a services ecosystem, as both a provider and a consumer, how should a curator evaluate service dependencies and their associated risks and benefits? Here are some guidelines. Many eggs, many baskets An instance of the calendar aggregator gathers events from three main sources: Eventful (service #1), Upcoming (service #2), and a curated set of iCalendar feeds. A subset of those feeds may (until recently) have been mediated by FuseCal (service #3). So there were three main service dependencies here, and that’s one form of diversification. But the iCalendar feeds represent another, and more powerful, form of diversification. One may be served up by a Drupal system, one may be an ICS file posted from Outlook 2007, one may be an instance of Google Calendar. Each depends on its own supporting services, but the ecosystem is very diverse. Data and service portability The elmcity project isn’t a database of events, but rather an aggregator of feeds of events. What matters in this case is portability of metadata describing the feeds, as well as data describing events. The system depends on Delicious for the management of the metadata. But all this metadata is replicated to Azure for safekeeping. Since the elmcity project does run on Azure, there’s clearly a strong dependence on that platform’s compute and storage services. But I could run the code on another host — even another cloud-based host, thanks to Amazon’s EC2 for Windows. Likewise I could store blobs and tables in Amazon’s S3 and SimpleDB. Strategic choices In this context, the use of FuseCal was a strategic choice. There isn’t a readily available replacement, and that’s a recipe for the sort of disruption we’ve just experienced. But since the system is diversified, that disruption is contained. Was the benefit provided by this unique service worth the cost of disruption? Some curators may disagree, but I think the answer is yes. It was really helpful to be able to show people that informational web pages are implicitly data feeds, and to show what can happen when those data feeds are made explicit. Still, it was a crutch. Ultimately we want people to stand on their own two feet, and take direct control of the information they publish to the web. FuseCal had to guess which times went with which events, and sometimes guessed wrong. If you’re publishing the event, you want to state these facts unambiguously. And using a variety of methods, as I’ve shown, you can. Those methods are the real strategic choices. If you can publish your own data feed, simply and inexpensively, you should seize the opportunity to do so Calendar pages successfully parsed by FuseCal prescottaz Events | Prescott Arizona News and Events Sharlot Hall Museum Events Board of Adjustments Board of Supervisors Prescott Public Library fallschurchcals City of Falls Church, VA Government Calendar City of Falls Church, VA Government Calendar City of Falls Church, VA Government Calendar The Dirty Pints – Traditional Irish Music – CALENDAR City of Falls Church Concert Band Dogfish Head Ale House Calendar ottawacals LibraryThing – Events within 25 miles of ottawa,on snoqualmie Meadowbrook Farm Classes and Events mashablecity RI Council for the Humanities elmcity The Greater Keene Men’s Softball Association Cheshire Soccer Club Jatoba in Keene Garrin Benfield in Keene Harlow’s Pub Kiwanis Events Kiwanis Meetings Monadnock Speedway Stonewall Farm Weathervane Music Hall, Brattleboro Mariposa Museum, Peterborough McCue’s Keene Cal Ripken Baseball Lonesome Lake: Jeffrey James Murphy, Bjorn Delacruz, Gregory George Moore Keene State College Antioch Cheshire Medical Center Monadnock Earth Festival Keene Community Master Plan a2cal LibraryThing events for Ann Arbor whyhuntington Cabell County Public Library, Huntington, WV Huntington Symphony Orchestra of Huntington, West Virginia St Mary’s Medical Center Cabell-Huntington Hospital Events Pullman Square Appalachian Film Festival Marshall University Department of Music Marshall Sports Create Huntington Big Sandy Superstore Arena Calendar Mountaineer Opry Arts Resources for the Tri State Paramount Arts Center (Ashland Ky) Calendar West Virginia Pumpkin Park Calendar/ Although I haven’t been able to confirm this officially yet, it looks like FuseCal, the HTML screen-scraping service that I’ve been using (and recommending) as a way to convert calendar-like web pages into iCalendar feeds, has shut down. The web pages that FuseCal has been successfully processing, for several curators participating in the elmcity project, are listed below. They’re a kind of existence proof, validating the notion that unstructured calendar info — what people intuitively create — can be mechanically transformed into structured info that syndicates reliably. I hope this service, or some future variant of it, will continue. It’s a really useful way to help people grasp the concept of publishing calendar feeds. But in the long run, it’s a set of training wheels. Ultimately we need to teach people why and how to produce such feeds more directly. All of the event information shown below could be managed in a more structured way using calendar software that produces data feeds for syndication and web pages for browsing. More broadly, incidents like this prompt us to consider the nature of the services ecosystem we’re all embedded in — as users and, increasingly, as co-creators. In the software business, developers have long since learned to evaluate the benefits and risks of “taking a dependency” on a component, library, or service. Users didn’t have to think too much about that. A software product that was discontinued would keep working perfectly well, maybe for years. But services can — and sometimes do — stop abruptly. Since the elmcity project is embedded in a services ecosystem, as both a provider and a consumer, how should a curator evaluate service dependencies and their associated risks and benefits? Here are some guidelines. Many eggs, many baskets An instance of the calendar aggregator gathers events from three main sources: Eventful (service #1), Upcoming (service #2), and a curated set of iCalendar feeds. A subset of those feeds may (until recently) have been mediated by FuseCal (service #3). So there were three main service dependencies here, and that’s one form of diversification. But the iCalendar feeds represent another, and more powerful, form of diversification. One may be served up by a Drupal system, one may be an ICS file posted from Outlook 2007, one may be an instance of Google Calendar. Each depends on its own supporting services, but the ecosystem is very diverse. Data and service portability The elmcity project isn’t a database of events, but rather an aggregator of feeds of events. What matters in this case is portability of metadata describing the feeds, as well as data describing events. The system depends on Delicious for the management of the metadata. But all this metadata is replicated to Azure for safekeeping. Since the elmcity project does run on Azure, there’s clearly a strong dependence on that platform’s compute and storage services. But I could run the code on another host — even another cloud-based host, thanks to Amazon’s EC2 for Windows. Likewise I could store blobs and tables in Amazon’s S3 and SimpleDB. Strategic choices In this context, the use of FuseCal was a strategic choice. There isn’t a readily available replacement, and that’s a recipe for the sort of disruption we’ve just experienced. But since the system is diversified, that disruption is contained. Was the benefit provided by this unique service worth the cost of disruption? Some curators may disagree, but I think the answer is yes. It was really helpful to be able to show people that informational web pages are implicitly data feeds, and to show what can happen when those data feeds are made explicit. Still, it was a crutch. Ultimately we want people to stand on their own two feet, and take direct control of the information they publish to the web. FuseCal had to guess which times went with which events, and sometimes guessed wrong. If you’re publishing the event, you want to state these facts unambiguously. And using a variety of methods, as I’ve shown, you can. Those methods are the real strategic choices. If you can publish your own data feed, simply and inexpensively, you should seize the opportunity to do so Calendar pages successfully parsed by FuseCal prescottaz Events | Prescott Arizona News and Events Sharlot Hall Museum Events Board of Adjustments Board of Supervisors Prescott Public Library fallschurchcals City of Falls Church, VA Government Calendar City of Falls Church, VA Government Calendar City of Falls Church, VA Government Calendar The Dirty Pints – Traditional Irish Music – CALENDAR City of Falls Church Concert Band Dogfish Head Ale House Calendar ottawacals LibraryThing – Events within 25 miles of ottawa,on snoqualmie Meadowbrook Farm Classes and Events mashablecity RI Council for the Humanities elmcity The Greater Keene Men’s Softball Association Cheshire Soccer Club Jatoba in Keene Garrin Benfield in Keene Harlow’s Pub Kiwanis Events Kiwanis Meetings Monadnock Speedway Stonewall Farm Weathervane Music Hall, Brattleboro Mariposa Museum, Peterborough McCue’s Keene Cal Ripken Baseball Lonesome Lake: Jeffrey James Murphy, Bjorn Delacruz, Gregory George Moore Keene State College Antioch Cheshire Medical Center Monadnock Earth Festival Keene Community Master Plan a2cal LibraryThing events for Ann Arbor whyhuntington Cabell County Public Library, Huntington, WV Huntington Symphony Orchestra of Huntington, West Virginia St Mary’s Medical Center Cabell-Huntington Hospital Events Pullman Square Appalachian Film Festival Marshall University Department of Music Marshall Sports Create Huntington Big Sandy Superstore Arena Calendar Mountaineer Opry Arts Resources for the Tri State Paramount Arts Center (Ashland Ky) Calendar West Virginia Pumpkin Park Calendar/ tag:odeo.com,2009-07-07,24764784 Tue, 07 Jul 2009 07:14:52 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24760549-Influencing-the-production-of-public-data In the latest installment of my Innovators podcast, which ran while I was away on vacation, I spoke with Steven Willmott of 3scale, one of several companies in the emerging business of third-party API management. As more organizations get into the game of providing APIs to their online data, there’s a growing need for help in the design and management of those APIs. By way of demonstration, 3scale is providing an unofficial API to some of the datasets offered by the United Nations. The UN data at http://data.un.org, while browseable and downloadable, is not programmatically accessible. If you visit 3scale’s demo at www.undata-api.org/ you can sign up for an access key, ask for available datasets — mostly, so far, from the World Health Organization (see below) — and then query them. The query capability is rather limited. For a given measure, like Births by caesarean section (percent), you can select subsets by country or by year, but you can’t query or ... In the latest installment of my Innovators podcast, which ran while I was away on vacation, I spoke with Steven Willmott of 3scale, one of several companies in the emerging business of third-party API management. As more organizations get into the game of providing APIs to their online data, there’s a growing need for help in the design and management of those APIs. By way of demonstration, 3scale is providing an unofficial API to some of the datasets offered by the United Nations. The UN data at http://data.un.org, while browseable and downloadable, is not programmatically accessible. If you visit 3scale’s demo at www.undata-api.org/ you can sign up for an access key, ask for available datasets — mostly, so far, from the World Health Organization (see below) — and then query them. The query capability is rather limited. For a given measure, like Births by caesarean section (percent), you can select subsets by country or by year, but you can’t query or order by values. And you can’t make correlations across tables in one query. It’s just a demo, of course. If 3scale wanted to invest more effort, a more robust query system could be built. The fact that such a system can be built by an unofficial intermediary, rather than by the provider of the data, is quite interesting. As I watch this data publication meme spread, here’s something that interests me even more. These efforts don’t really reflect the Web 2.0 values of engagement and participation to the extent they could. We’re now very focused on opening up flexible means of access to data. But the conversation is still framed in terms of a producer/consumer relationship that isn’t itself much discussed. At the end of this entry you’ll find a list of WHO datasets. Here’s one: Community and traditional health workers density (per 10,000 population). What kinds of questions do we think we might try to answer by counting this category of worker? What kinds of questions can’t we try to answer using the datasets WHO is collecting? How might we therefore want to try to influence the WHO’s data-gathering efforts, and those of other public health organizations? “Give us the data” is an easy slogan to chant. And there’s no doubt that much good will come from poking through what we are given. But we also need to have ideas about what we want the data for, and communicate those ideas to the providers who are gathering it on our behalf. Adolescent fertility rate Adult literacy rate (percent) Gross national income per capita (PPP international $) Net primary school enrolment ratio female (percent) Net primary school enrolment ratio male (percent) Population (in thousands) total Population annual growth rate (percent) Population in urban areas (percent) Population living below the poverty line (percent living on less than US$1 per day) Population median age (years) Population proportion over 60 (percent) Population proportion under 15 (percent) Registration coverage of births (percent) Registration coverage of deaths (percent) Total fertility rate (per woman) Antenatal care coverage – at least four visits (percent) Antiretroviral therapy coverage among HIV-infected pregnant women for PMTCT (percent) Antiretroviral therapy coverage among people with advanced HIV infections (percent) Births attended by skilled health personnel (percent) Births by caesarean section (percent) Children aged 6-59 months who received vitamin A supplementation (percent) Children aged less than 5 years sleeping under insecticide-treated nets (percent) Children aged less than 5 years who received any antimalarial treatment for fever (percent) Children aged less than 5 years with ARI symptoms taken to facility (percent) Children aged less than 5 years with diarrhoea receiving ORT (percent) Contraceptive prevalence (percent) Neonates protected at birth against neonatal tetanus (PAB) (percent) One-year-olds immunized with MCV One-year-olds immunized with three doses of Hepatitis B (HepB3) (percent) One-year-olds immunized with three doses of Hib (Hib3) vaccine (percent) One-year-olds immunized with three doses of diphtheria tetanus toxoid and pertussis (DTP3) (percent) Tuberculosis detection rate under DOTS (percent) Tuberculosis treatment success under DOTS (percent) Women who have had PAP smear (percent) Women who have had mammography (percent) Community and traditional health workers density (per 10 000 population) Dentistry personnel density (per 10 000 population) Environment and public health workers density (per 10 000 population) External resources for health as percentage of total expenditure on health General government expenditure on health as percentage of total expenditure on health General government expenditure on health as percentage of total government expenditure Hospital beds (per 10 000 population) Laboratory health workers density (per 10 000 population) Number of community and traditional health workers Number of dentistry personnel Number of environment and public health workers Number of laboratory health workers Number of nursing and midwifery personnel Number of other health service providers Number of pharmaceutical personnel Nursing and midwifery personnel density (per 10 000 population) Other health service providers density (per 10 000 population) Out-of-pocket expenditure as percentage of private expenditure on health Per capita total expenditure on health (PPP int. $) Per capita total expenditure on health at average exchange rate (US$ Pharmaceutical personnel density (per 10 000 population) Physicians density (per 10 000 population) Private expenditure on health as percentage of total expenditure on health Private prepaid plans as percentage of private expenditure on health Ratio of health management and support workers to health service providers Ratio of nurses and midwives to physicians Social security expenditure on health as percentage of general government expenditure on health Total expenditure on health as percentage of gross domestic product In the latest installment of my Innovators podcast, which ran while I was away on vacation, I spoke with Steven Willmott of 3scale, one of several companies in the emerging business of third-party API management. As more organizations get into the game of providing APIs to their online data, there’s a growing need for help in the design and management of those APIs. By way of demonstration, 3scale is providing an unofficial API to some of the datasets offered by the United Nations. The UN data at http://data.un.org, while browseable and downloadable, is not programmatically accessible. If you visit 3scale’s demo at www.undata-api.org/ you can sign up for an access key, ask for available datasets — mostly, so far, from the World Health Organization (see below) — and then query them. The query capability is rather limited. For a given measure, like Births by caesarean section (percent), you can select subsets by country or by year, but you can’t query or order by values. And you can’t make correlations across tables in one query. It’s just a demo, of course. If 3scale wanted to invest more effort, a more robust query system could be built. The fact that such a system can be built by an unofficial intermediary, rather than by the provider of the data, is quite interesting. As I watch this data publication meme spread, here’s something that interests me even more. These efforts don’t really reflect the Web 2.0 values of engagement and participation to the extent they could. We’re now very focused on opening up flexible means of access to data. But the conversation is still framed in terms of a producer/consumer relationship that isn’t itself much discussed. At the end of this entry you’ll find a list of WHO datasets. Here’s one: Community and traditional health workers density (per 10,000 population). What kinds of questions do we think we might try to answer by counting this category of worker? What kinds of questions can’t we try to answer using the datasets WHO is collecting? How might we therefore want to try to influence the WHO’s data-gathering efforts, and those of other public health organizations? “Give us the data” is an easy slogan to chant. And there’s no doubt that much good will come from poking through what we are given. But we also need to have ideas about what we want the data for, and communicate those ideas to the providers who are gathering it on our behalf. Adolescent fertility rate Adult literacy rate (percent) Gross national income per capita (PPP international $) Net primary school enrolment ratio female (percent) Net primary school enrolment ratio male (percent) Population (in thousands) total Population annual growth rate (percent) Population in urban areas (percent) Population living below the poverty line (percent living on less than US$1 per day) Population median age (years) Population proportion over 60 (percent) Population proportion under 15 (percent) Registration coverage of births (percent) Registration coverage of deaths (percent) Total fertility rate (per woman) Antenatal care coverage – at least four visits (percent) Antiretroviral therapy coverage among HIV-infected pregnant women for PMTCT (percent) Antiretroviral therapy coverage among people with advanced HIV infections (percent) Births attended by skilled health personnel (percent) Births by caesarean section (percent) Children aged 6-59 months who received vitamin A supplementation (percent) Children aged less than 5 years sleeping under insecticide-treated nets (percent) Children aged less than 5 years who received any antimalarial treatment for fever (percent) Children aged less than 5 years with ARI symptoms taken to facility (percent) Children aged less than 5 years with diarrhoea receiving ORT (percent) Contraceptive prevalence (percent) Neonates protected at birth against neonatal tetanus (PAB) (percent) One-year-olds immunized with MCV One-year-olds immunized with three doses of Hepatitis B (HepB3) (percent) One-year-olds immunized with three doses of Hib (Hib3) vaccine (percent) One-year-olds immunized with three doses of diphtheria tetanus toxoid and pertussis (DTP3) (percent) Tuberculosis detection rate under DOTS (percent) Tuberculosis treatment success under DOTS (percent) Women who have had PAP smear (percent) Women who have had mammography (percent) Community and traditional health workers density (per 10 000 population) Dentistry personnel density (per 10 000 population) Environment and public health workers density (per 10 000 population) External resources for health as percentage of total expenditure on health General government expenditure on health as percentage of total expenditure on health General government expenditure on health as percentage of total government expenditure Hospital beds (per 10 000 population) Laboratory health workers density (per 10 000 population) Number of community and traditional health workers Number of dentistry personnel Number of environment and public health workers Number of laboratory health workers Number of nursing and midwifery personnel Number of other health service providers Number of pharmaceutical personnel Nursing and midwifery personnel density (per 10 000 population) Other health service providers density (per 10 000 population) Out-of-pocket expenditure as percentage of private expenditure on health Per capita total expenditure on health (PPP int. $) Per capita total expenditure on health at average exchange rate (US$ Pharmaceutical personnel density (per 10 000 population) Physicians density (per 10 000 population) Private expenditure on health as percentage of total expenditure on health Private prepaid plans as percentage of private expenditure on health Ratio of health management and support workers to health service providers Ratio of nurses and midwives to physicians Social security expenditure on health as percentage of general government expenditure on health Total expenditure on health as percentage of gross domestic product tag:odeo.com,2009-07-06,24760549 Mon, 06 Jul 2009 07:21:26 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24745657-New-England-still-too-wet-Escaping-to-sunny-Old-England We’re just back from a Caribbean vacation — with a couple of interesting souvenirs in tow. Under normal circumstances I’d feel a twinge of regret about turning around a day later and heading out again. But I’m not really in the mood to build an ark, which after 40 days of rain is about to become the new summer sport here in New England. And while the wet isn’t letting up yet here, the weather looks lovely in Old England. So it’s actually a great time to head off to London for a Tuesday visit and talk at Nature Publishing, panels and a talk at the Activate conference on Wednesday, and another talk at the Guardian on Thursday. That one is open to the guests — for the first time, I gather. The writeup also notes: Many people will then head down to the Rotunda bar for drinks on the canal waterfront after the talk at about 6. In all these venues I’ll be expanding on the themes I’ve written about here lately: collaborative curation, co... We’re just back from a Caribbean vacation — with a couple of interesting souvenirs in tow. Under normal circumstances I’d feel a twinge of regret about turning around a day later and heading out again. But I’m not really in the mood to build an ark, which after 40 days of rain is about to become the new summer sport here in New England. And while the wet isn’t letting up yet here, the weather looks lovely in Old England. So it’s actually a great time to head off to London for a Tuesday visit and talk at Nature Publishing, panels and a talk at the Activate conference on Wednesday, and another talk at the Guardian on Thursday. That one is open to the guests — for the first time, I gather. The writeup also notes: Many people will then head down to the Rotunda bar for drinks on the canal waterfront after the talk at about 6. In all these venues I’ll be expanding on the themes I’ve written about here lately: collaborative curation, computational thinking for everyone, community calendars as a motivating case study, and Azure as platform for doing stuff in the cloud. By the time I get home for July 4, it ought to be dry here. If not, I’ll break out my cubit-calibrated tape measure and get to work on that ark. We’re just back from a Caribbean vacation — with a couple of interesting souvenirs in tow. Under normal circumstances I’d feel a twinge of regret about turning around a day later and heading out again. But I’m not really in the mood to build an ark, which after 40 days of rain is about to become the new summer sport here in New England. And while the wet isn’t letting up yet here, the weather looks lovely in Old England. So it’s actually a great time to head off to London for a Tuesday visit and talk at Nature Publishing, panels and a talk at the Activate conference on Wednesday, and another talk at the Guardian on Thursday. That one is open to the guests — for the first time, I gather. The writeup also notes: Many people will then head down to the Rotunda bar for drinks on the canal waterfront after the talk at about 6. In all these venues I’ll be expanding on the themes I’ve written about here lately: collaborative curation, computational thinking for everyone, community calendars as a motivating case study, and Azure as platform for doing stuff in the cloud. By the time I get home for July 4, it ought to be dry here. If not, I’ll break out my cubit-calibrated tape measure and get to work on that ark. tag:odeo.com,2009-06-29,24745657 Mon, 29 Jun 2009 08:39:38 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24724882-It%E2%80%99s-the-headings-stupid My recent adventure in naming the times of day was so much fun that I lost track of the original purpose of the exercise, which was to improve accessibility for sight-impaired users. When I interpersed time-of-day labels into each day’s event listing, I used HTML DIV tags. Wrong, wrong, wrong! Those labels are structural elements, and as my accessibility consultant Susan Gerhart gently reminded me, screen readers depend on HTML headings to find and announce them. The labels should have been second-level headings — i.e., HTML H2 tags. It gets worse. When Susan prompted me to take another look at what I’d done, I found that the date labels were inexplicably tagged as paragraphs (P) instead of the top-level headers (H1) that they logically are. Oh. Right. Of course. Duh. Fixed. Sorry. What was I thinking? How could somebody like me, who has preached about the attention-focusing power of heads, decks, and leads, screw up something so basic as this? Easily, as it turns ... My recent adventure in naming the times of day was so much fun that I lost track of the original purpose of the exercise, which was to improve accessibility for sight-impaired users. When I interpersed time-of-day labels into each day’s event listing, I used HTML DIV tags. Wrong, wrong, wrong! Those labels are structural elements, and as my accessibility consultant Susan Gerhart gently reminded me, screen readers depend on HTML headings to find and announce them. The labels should have been second-level headings — i.e., HTML H2 tags. It gets worse. When Susan prompted me to take another look at what I’d done, I found that the date labels were inexplicably tagged as paragraphs (P) instead of the top-level headers (H1) that they logically are. Oh. Right. Of course. Duh. Fixed. Sorry. What was I thinking? How could somebody like me, who has preached about the attention-focusing power of heads, decks, and leads, screw up something so basic as this? Easily, as it turns out, in the absence of feedback. If you yourself don’t depend on a design feature, there is a natural tendency to forget why it matters to others. Coincidentally (or not) Susan recently wrote an essay, and published a companion audio recording, that will help me — and I hope others — not to forget again. Entitled Hear Me Stumble Around White House, Recovery, and Data GOV web sites, it’s a blow-by-blow account of her efforts to navigate those sites with a screen reader. In this recording you can hear Susan and her screen reader trying to make sense of whitehouse.gov. If you’ve never heard a screen reader in action, it’s worth listening for that alone. You’ll get a very clear sense of how these tools depend on the hierarchy of the page. Simultaneously you’ll hear Susan narrate her intention — to read an article about cybersecurity — and her frustration. For example: I was thrown off by the slide show at the top of the page. Once I hit the cybersecurity story, the next time I traverse this section the story was about the Supreme Court nominee. Despite this randomness, the page does at least identify the top stories with H1 tags. And Signed Legislation is an H2. But none of the headlines under Signed Legislation are H3s, they’re Ps. Over at recovery.gov and data.gov Susan finds none at all, and reacts to their omissions less gently she did to mine: It’s the headings, stupid!!! Thanks. I will try not to forget that again. PS: In a follow-up to her blog essay, Susan links to detailed reports by accessibility pioneer Jim Thatcher on the issues he found with data.gov and recovery.gov. My recent adventure in naming the times of day was so much fun that I lost track of the original purpose of the exercise, which was to improve accessibility for sight-impaired users. When I interpersed time-of-day labels into each day’s event listing, I used HTML DIV tags. Wrong, wrong, wrong! Those labels are structural elements, and as my accessibility consultant Susan Gerhart gently reminded me, screen readers depend on HTML headings to find and announce them. The labels should have been second-level headings — i.e., HTML H2 tags. It gets worse. When Susan prompted me to take another look at what I’d done, I found that the date labels were inexplicably tagged as paragraphs (P) instead of the top-level headers (H1) that they logically are. Oh. Right. Of course. Duh. Fixed. Sorry. What was I thinking? How could somebody like me, who has preached about the attention-focusing power of heads, decks, and leads, screw up something so basic as this? Easily, as it turns out, in the absence of feedback. If you yourself don’t depend on a design feature, there is a natural tendency to forget why it matters to others. Coincidentally (or not) Susan recently wrote an essay, and published a companion audio recording, that will help me — and I hope others — not to forget again. Entitled Hear Me Stumble Around White House, Recovery, and Data GOV web sites, it’s a blow-by-blow account of her efforts to navigate those sites with a screen reader. In this recording you can hear Susan and her screen reader trying to make sense of whitehouse.gov. If you’ve never heard a screen reader in action, it’s worth listening for that alone. You’ll get a very clear sense of how these tools depend on the hierarchy of the page. Simultaneously you’ll hear Susan narrate her intention — to read an article about cybersecurity — and her frustration. For example: I was thrown off by the slide show at the top of the page. Once I hit the cybersecurity story, the next time I traverse this section the story was about the Supreme Court nominee. Despite this randomness, the page does at least identify the top stories with H1 tags. And Signed Legislation is an H2. But none of the headlines under Signed Legislation are H3s, they’re Ps. Over at recovery.gov and data.gov Susan finds none at all, and reacts to their omissions less gently she did to mine: It’s the headings, stupid!!! Thanks. I will try not to forget that again. PS: In a follow-up to her blog essay, Susan links to detailed reports by accessibility pioneer Jim Thatcher on the issues he found with data.gov and recovery.gov. tag:odeo.com,2009-06-18,24724882 Thu, 18 Jun 2009 06:37:07 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24719547-Endangered-languages-and-linguistic-best-practices Daniel Everett’s recent Long Now talk about endangered languages (writeup, mp3) includes this gem reported by Stewart Brand: Among other things, the wide variety of verb forms are used to account for the directness of evidence for a statement. Everett originally went to the Pirahã in 1977 as a Christian missionary. They challenged him to provide evidence for the existence of Jesus, and lost interest when he couldn’t. Eventually so did he. The Pirahã made him an atheist. This is so interesting that it’s worth unpacking for those who won’t have time to listen. Among the sixteen suffixes for verbs, there are three that convey the source of evidence: I heard that Dan went fishing. I saw Dan go fishing. I deduce, from the available evidence, that Dan went fishing. These assertions might not be true. The Pirahã, being human, do sometimes lie. But I love the idea of a culture in which evidence-based thinking is baked into the language. There are only a few hundred P... Daniel Everett’s recent Long Now talk about endangered languages (writeup, mp3) includes this gem reported by Stewart Brand: Among other things, the wide variety of verb forms are used to account for the directness of evidence for a statement. Everett originally went to the Pirahã in 1977 as a Christian missionary. They challenged him to provide evidence for the existence of Jesus, and lost interest when he couldn’t. Eventually so did he. The Pirahã made him an atheist. This is so interesting that it’s worth unpacking for those who won’t have time to listen. Among the sixteen suffixes for verbs, there are three that convey the source of evidence: I heard that Dan went fishing. I saw Dan go fishing. I deduce, from the available evidence, that Dan went fishing. These assertions might not be true. The Pirahã, being human, do sometimes lie. But I love the idea of a culture in which evidence-based thinking is baked into the language. There are only a few hundred Pirahã, and their language is only one of thousands — more than half unwritten — that are endangered. The talk ends with plea to preserve and document those languages. It has never been easier to capture and disseminate recorded audio, or to collaboratively curate such material, so I hope these capabilities will be put to good use in the quest to preserve linguistic diversity. But no matter what, we’re going to continue to lose languages. Maybe, though, if we can identify some of the ways of thinking encoded in those languages, we can carry them forward. Respect for the source of evidence is a great example. I could have simply told you about what Daniel Everett said, and what Stewart Brand wrote about what Daniel Everett said. But it was possible to form links to the audio and text, so I did. I wonder how many other best practices are encoded in those thousands of endangered languages. And I wonder if it might be possible to identify and catalog more of them. Daniel Everett’s recent Long Now talk about endangered languages (writeup, mp3) includes this gem reported by Stewart Brand: Among other things, the wide variety of verb forms are used to account for the directness of evidence for a statement. Everett originally went to the Pirahã in 1977 as a Christian missionary. They challenged him to provide evidence for the existence of Jesus, and lost interest when he couldn’t. Eventually so did he. The Pirahã made him an atheist. This is so interesting that it’s worth unpacking for those who won’t have time to listen. Among the sixteen suffixes for verbs, there are three that convey the source of evidence: I heard that Dan went fishing. I saw Dan go fishing. I deduce, from the available evidence, that Dan went fishing. These assertions might not be true. The Pirahã, being human, do sometimes lie. But I love the idea of a culture in which evidence-based thinking is baked into the language. There are only a few hundred Pirahã, and their language is only one of thousands — more than half unwritten — that are endangered. The talk ends with plea to preserve and document those languages. It has never been easier to capture and disseminate recorded audio, or to collaboratively curate such material, so I hope these capabilities will be put to good use in the quest to preserve linguistic diversity. But no matter what, we’re going to continue to lose languages. Maybe, though, if we can identify some of the ways of thinking encoded in those languages, we can carry them forward. Respect for the source of evidence is a great example. I could have simply told you about what Daniel Everett said, and what Stewart Brand wrote about what Daniel Everett said. But it was possible to form links to the audio and text, so I did. I wonder how many other best practices are encoded in those thousands of endangered languages. And I wonder if it might be possible to identify and catalog more of them. tag:odeo.com,2009-06-17,24719547 Wed, 17 Jun 2009 06:24:14 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24709020-When-does-afternoon-begin When I invited folks to become calendar curators for the elmcity project, the person who stepped forward in Prescott AZ was Susan Gerhart, whom I interviewed here. One of her great insights about web design is that the right thing for a vision-impaired user is almost always also the right thing for everyone. She calls this the curb cuts principle: Curb cuts for wheelchairs also guide blind persons into street crossings and prevent accidents for baby strollers, bicyclists, skateboarders, and inattentive walkers. So I shouldn’t have been surprised when Susan noticed that the HTML rendering of the calendar need some curb cuts. Within each day, the events show up as a long undifferentiated list. She suggested that subdividing the list by time of day — morning, afternoon, evening — will be helpful to folks using screen readers. But in fact, it’s just plain helpful. So I’m testing a version of that idea now. Ionically I was just thinking about this same princ... When I invited folks to become calendar curators for the elmcity project, the person who stepped forward in Prescott AZ was Susan Gerhart, whom I interviewed here. One of her great insights about web design is that the right thing for a vision-impaired user is almost always also the right thing for everyone. She calls this the curb cuts principle: Curb cuts for wheelchairs also guide blind persons into street crossings and prevent accidents for baby strollers, bicyclists, skateboarders, and inattentive walkers. So I shouldn’t have been surprised when Susan noticed that the HTML rendering of the calendar need some curb cuts. Within each day, the events show up as a long undifferentiated list. She suggested that subdividing the list by time of day — morning, afternoon, evening — will be helpful to folks using screen readers. But in fact, it’s just plain helpful. So I’m testing a version of that idea now. Ionically I was just thinking about this same principle in another context. The new version of Oakland Crimespotting, which I raved about, segments incidents using this vocabulary: light, dark, commute, nightlife, day, night, swing shift In that spirit, I’m trying this: morning, lunch, afternoon, evening, night This of course leads to the question: When do these times begin and end? I was fascinated to see that both Google and Bing return the same Yahoo answers page for the query morning afternoon evening. For now, though, I’m going with this ruleset: Morning: 5:00 AM to 11:30 AM Lunch: 11:30 AM to 1:00 PM Afternoon: 1:30 PM to 5:30 PM Evening: 5:30 PM to 9:00 PM Night: 9:00 PM to 5:00 AM But I’ll make these rules — and maybe even the time-of-day names — configurable on a per-location basis. When I invited folks to become calendar curators for the elmcity project, the person who stepped forward in Prescott AZ was Susan Gerhart, whom I interviewed here. One of her great insights about web design is that the right thing for a vision-impaired user is almost always also the right thing for everyone. She calls this the curb cuts principle: Curb cuts for wheelchairs also guide blind persons into street crossings and prevent accidents for baby strollers, bicyclists, skateboarders, and inattentive walkers. So I shouldn’t have been surprised when Susan noticed that the HTML rendering of the calendar need some curb cuts. Within each day, the events show up as a long undifferentiated list. She suggested that subdividing the list by time of day — morning, afternoon, evening — will be helpful to folks using screen readers. But in fact, it’s just plain helpful. So I’m testing a version of that idea now. Ionically I was just thinking about this same principle in another context. The new version of Oakland Crimespotting, which I raved about, segments incidents using this vocabulary: light, dark, commute, nightlife, day, night, swing shift In that spirit, I’m trying this: morning, lunch, afternoon, evening, night This of course leads to the question: When do these times begin and end? I was fascinated to see that both Google and Bing return the same Yahoo answers page for the query morning afternoon evening. For now, though, I’m going with this ruleset: Morning: 5:00 AM to 11:30 AM Lunch: 11:30 AM to 1:00 PM Afternoon: 1:30 PM to 5:30 PM Evening: 5:30 PM to 9:00 PM Night: 9:00 PM to 5:00 AM But I’ll make these rules — and maybe even the time-of-day names — configurable on a per-location basis. tag:odeo.com,2009-06-15,24709020 Mon, 15 Jun 2009 13:44:10 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24685225-Bulk-search-and-replace-for-blog-entries Last night I realized there was one more step needed to restore my 2002-2006 archive. All of my references into that archive from this blog, which started in December 2006, had to be redirected. What’s more, they had to be remapped. Old URLs like http://weblog.infoworld.com/udell/2006/12/04.html#a1571 had to become new URLs like http://jonudell.net/udell/2006-12-04-hunting-the-elusive-search-strategy.html. Even without the remapping, it’s not obvious how to do a simple search and replace (say, from weblog.infoworld.com/udell to jonudell.net/udell) across a set of blog entries. I tried the export/edit/import route, but — at least in the case of WordPress — that doesn’t seem to be a way to update existing stuff. So I wound up writing a script that uses the MetaWeblog API to fetch my current blog entries, find references to the old namespace, adjust them to point to the new namespace, and update the entries. It’s here for my own future reference, and... Last night I realized there was one more step needed to restore my 2002-2006 archive. All of my references into that archive from this blog, which started in December 2006, had to be redirected. What’s more, they had to be remapped. Old URLs like http://weblog.infoworld.com/udell/2006/12/04.html#a1571 had to become new URLs like http://jonudell.net/udell/2006-12-04-hunting-the-elusive-search-strategy.html. Even without the remapping, it’s not obvious how to do a simple search and replace (say, from weblog.infoworld.com/udell to jonudell.net/udell) across a set of blog entries. I tried the export/edit/import route, but — at least in the case of WordPress — that doesn’t seem to be a way to update existing stuff. So I wound up writing a script that uses the MetaWeblog API to fetch my current blog entries, find references to the old namespace, adjust them to point to the new namespace, and update the entries. It’s here for my own future reference, and for yours if you need it. As always in these situations, I end up wondering what a civilian would do. Blog publishing systems don’t seem have bulk search-and-replace capability. They do, however, have APIs. There could be a tool or service that helps people make these kinds of changes. It’d be hard to avoid the password anti-pattern, so if this were a cloud-based service rather than a locally-installed tool you’d want to change your password after using it. But still, it should be doable. Do such tools or services exist? Last night I realized there was one more step needed to restore my 2002-2006 archive. All of my references into that archive from this blog, which started in December 2006, had to be redirected. What’s more, they had to be remapped. Old URLs like http://weblog.infoworld.com/udell/2006/12/04.html#a1571 had to become new URLs like http://jonudell.net/udell/2006-12-04-hunting-the-elusive-search-strategy.html. Even without the remapping, it’s not obvious how to do a simple search and replace (say, from weblog.infoworld.com/udell to jonudell.net/udell) across a set of blog entries. I tried the export/edit/import route, but — at least in the case of WordPress — that doesn’t seem to be a way to update existing stuff. So I wound up writing a script that uses the MetaWeblog API to fetch my current blog entries, find references to the old namespace, adjust them to point to the new namespace, and update the entries. It’s here for my own future reference, and for yours if you need it. As always in these situations, I end up wondering what a civilian would do. Blog publishing systems don’t seem have bulk search-and-replace capability. They do, however, have APIs. There could be a tool or service that helps people make these kinds of changes. It’d be hard to avoid the password anti-pattern, so if this were a cloud-based service rather than a locally-installed tool you’d want to change your password after using it. But still, it should be doable. Do such tools or services exist? tag:odeo.com,2009-06-10,24685225 Wed, 10 Jun 2009 09:20:04 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24681952-Rebooting-my-2002-2006-archive While spot-checking my mostly-reconstructed 2002-2006 blog, I found this plaint from 2002: When you are a writer whose entire corpus exists online, woven into a fabric of citation and commentary, it is incredibly painful to see that fabric torn apart. Déjà vu all over again. In 2002 I had to sacrifice the linkage to my 1999-2002 BYTE.com and restore it here. Now I’ve done the same for my 2002-2006 InfoWorld blog. Since its former namespace isn’t being redirected, and since all the old links were broken anyway, I’ve taken this opportunity to create new descriptive names that incorporate dates and titles. The reboot isn’t 100% clean, but it’s automated and reproducible so I can address categories of problems as they show up. I’m glad I’m not in publishing anymore. It turns out to be a lousy way to keep your stuff published. When a commercial hosted lifebits service comes online, I’ll be customer #1. While spot-checking my mostly-reconstructed 2002-2006 blog, I found this plaint from 2002: When you are a writer whose entire corpus exists online, woven into a fabric of citation and commentary, it is incredibly painful to see that fabric torn apart. Déjà vu all over again. In 2002 I had to sacrifice the linkage to my 1999-2002 BYTE.com and restore it here. Now I’ve done the same for my 2002-2006 InfoWorld blog. Since its former namespace isn’t being redirected, and since all the old links were broken anyway, I’ve taken this opportunity to create new descriptive names that incorporate dates and titles. The reboot isn’t 100% clean, but it’s automated and reproducible so I can address categories of problems as they show up. I’m glad I’m not in publishing anymore. It turns out to be a lousy way to keep your stuff published. When a commercial hosted lifebits service comes online, I’ll be customer #1. While spot-checking my mostly-reconstructed 2002-2006 blog, I found this plaint from 2002: When you are a writer whose entire corpus exists online, woven into a fabric of citation and commentary, it is incredibly painful to see that fabric torn apart. Déjà vu all over again. In 2002 I had to sacrifice the linkage to my 1999-2002 BYTE.com and restore it here. Now I’ve done the same for my 2002-2006 InfoWorld blog. Since its former namespace isn’t being redirected, and since all the old links were broken anyway, I’ve taken this opportunity to create new descriptive names that incorporate dates and titles. The reboot isn’t 100% clean, but it’s automated and reproducible so I can address categories of problems as they show up. I’m glad I’m not in publishing anymore. It turns out to be a lousy way to keep your stuff published. When a commercial hosted lifebits service comes online, I’ll be customer #1. tag:odeo.com,2009-06-09,24681952 Tue, 09 Jun 2009 21:13:03 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24676912-London-July-1-Activate On July 1 I’ll be in London to speak at the Guardian’s Activate summit. In a writeup on his blog, my former InfoWorld colleague Matt McAlister sets the tone: What does the Internet mean? How far will the changes go? Which aspects of civilization itself will become something different, perhaps even unrecognizable to us today through the pervasive effect of the network? It looks to be the kind of diverse gathering that really charges my batteries. In an interview published today, I sketch the themes of my talk: collaborative curation, social information management, syndication. These principles and practices are at the core of the elmcity project. I think they’re also at the core of the civilizational change that pervasive networking can bring. On July 1 I’ll be in London to speak at the Guardian’s Activate summit. In a writeup on his blog, my former InfoWorld colleague Matt McAlister sets the tone: What does the Internet mean? How far will the changes go? Which aspects of civilization itself will become something different, perhaps even unrecognizable to us today through the pervasive effect of the network? It looks to be the kind of diverse gathering that really charges my batteries. In an interview published today, I sketch the themes of my talk: collaborative curation, social information management, syndication. These principles and practices are at the core of the elmcity project. I think they’re also at the core of the civilizational change that pervasive networking can bring. On July 1 I’ll be in London to speak at the Guardian’s Activate summit. In a writeup on his blog, my former InfoWorld colleague Matt McAlister sets the tone: What does the Internet mean? How far will the changes go? Which aspects of civilization itself will become something different, perhaps even unrecognizable to us today through the pervasive effect of the network? It looks to be the kind of diverse gathering that really charges my batteries. In an interview published today, I sketch the themes of my talk: collaborative curation, social information management, syndication. These principles and practices are at the core of the elmcity project. I think they’re also at the core of the civilizational change that pervasive networking can bring. tag:odeo.com,2009-06-09,24676912 Tue, 09 Jun 2009 09:58:33 -0700 no Jon Udell Uncategorized http://odeo.com/episodes/24669298-Rebooting-my-2002-2006-archive While spot-checking my mostly-reconstructed 2002-2006 blog, I found this plaint from 2002: When you are a writer whose entire corpus exists online, woven into a fabric of citation and commentary, it is incredibly painful to see that fabric torn apart. Déjà vu all over again. In 2002 I had to sacrifice the linkage to my 1999-2002 BYTE.com and restore it here. Now I’ve done the same for my 2002-2006 InfoWorld blog. Since its former namespace isn’t being redirected, and since all the old links were broken anyway, I’ve taken this opportunity to create new descriptive names that incorporate dates and titles. The reboot isn’t 100% clean, but it’s automated and reproducible so I can address categories of problems as they show up. I’m glad I’m not in publishing anymore. It turns out to be a lousy way to keep your stuff published. When a commercial hosted lifebits service comes online, I’ll be customer #1. While spot-checking my mostly-reconstructed 2002-2006 blog, I found this plaint from 2002: When you are a writer whose entire corpus exists online, woven into a fabric of citation and commentary, it is incredibly painful to see that fabric torn apart. Déjà vu all over again. In 2002 I had to sacrifice the linkage to my 1999-2002 BYTE.com and restore it here. Now I’ve done the same for my 2002-2006 InfoWorld blog. Since its former namespace isn’t being redirected, and since all the old links were broken anyway, I’ve taken this opportunity to create new descriptive names that incorporate dates and titles. The reboot isn’t 100% clean, but it’s automated and reproducible so I can address categories of problems as they show up. I’m glad I’m not in publishing anymore. It turns out to be a lousy way to keep your stuff published. When a commercial hosted lifebits service comes online, I’ll be customer #1. While spot-checking my mostly-reconstructed 2002-2006 blog, I found this plaint from 2002: When you are a writer whose entire corpus exists online, woven into a fabric of citation and commentary, it is incredibly painful to see that fabric torn apart. Déjà vu all over again. In 2002 I had to sacrifice the linkage to my 1999-2002 BYTE.com and restore it here. Now I’ve done the same for my 2002-2006 InfoWorld blog. Since its former namespace isn’t being redirected, and since all the old links were broken anyway, I’ve taken this opportunity to create new descriptive names that incorporate dates and titles. The reboot isn’t 100% clean, but it’s automated and reproducible so I can address categories of problems as they show up. I’m glad I’m not in publishing anymore. It turns out to be a lousy way to keep your stuff published. When a commercial hosted lifebits service comes online, I’ll be customer #1. tag:odeo.com,2009-06-08,24669298 Mon, 08 Jun 2009 21:54:50 -0700 no Jon Udell Uncategorized