People upload thousands of pictures to Flickr every day, but the numbers and rates don't give the picture count justice. For the Future of Photography Museum in Amsterdam, Erik Kessels printed 24 hours of Flickr photos:

As you might imagine, this results in a lot of images, that fill the gallery space in an avalanche of photos. "We're exposed to an overload of images nowadays," says Kessels. "This glut is in large part the result of image-sharing sites like Flickr, networking sites like Facebook, and picture-based search engines. Their content mingles public and private, with the very personal being openly and un-selfconsciously displayed. By printing all the images uploaded in a 24-hour period, I visualise the feeling of drowning in representations of other peoples' experiences."

[Creative Review via Waxy]

Eric Fischer, who continues his string of mapping fun and doesn't even do it for his day job, maps the world in binary subdivisions, based on geotagged tweets. Each bounding box contains and equal number of geotagged tweets.

The best part is that Fischer is actually doing some problem-solving, trying to figure something out, so it's not just a pretty picture.

The actual motivation behind it, by the way, was to figure out an approximately optimal set of bounding boxes to query for in APIs like Picasa's, where if you ask for the whole world, you only get a few, very recent, results, but if you query for small enough bounding boxes, you can see further back in time. The idea is to choose bounding boxes with equal frequency so you get approximately the same time period of results from each of them.

Here's the image zoomed in on the United States. As you'd expect, it the concentration of boxes looks a lot like population density:

As does the view of Europe:

More maps from Fischer here.

[Binary subdivision of the world via @datapointed]

For all the maps by Eric Fischer I've posted, it's amazing how little I actually know about him, but in his most recent series, See Something or Say Something, he places geocoded tweets and Flickr photos on the same map. Blue dots represent tweets with location and orange dots are Flickr photos. White dots are locations with both.

Here's a zoomed in view of Europe:

And the United States:

Not surprisingly, you see a lot of white dots at city centers. That's an artifact of population density and Flickr and Twitter users. What's more interesting though are the areas outside of the city dominated by blue and orange. For example, the in the US map above, the east is dominated by blue, whereas the west seems to be more orange.

What compels people to tweet over taking a picture and vice versa? Or are we just seeing a Twitter scrape that happened in the early morning, before the west coast woke up?

Although, photo-heavy places just might be places that have things that are, well, worth taking pictures of. This map of Manhattan, for example, follows a similar pattern to Fischer's previous map on where tourists flock.

Here's one more for Bangkok. Clear split between tweets and photos. I like how it's fine grained enough we start to see roads.

See the full set of beautiful maps on Flickr.

Oh, and in case you're wondering how Fischer made these maps, here's how he did it:

There's not a whole lot of technology behind it. It's a C program that runs through the photos/tweets in chronological order, plotting the earliest ones the most brightly and stepping the brightness down for points that don't show up for the first time until later on. Points are also allowed to diffuse by a few pixels when there is an additional record for a point that is already plotted, with the brightness falling off exponentially as the point that is actually plotted gets further from its intended location. Each pixel is the somewhat weird area of 2.25 square miles because a smaller area made the whole-world image too big for Flickr to let me post it.

Simple idea. Good execution. Pretty results.

[See Something or Say Something via infosthetics]

read more

The trained guinea pigs in the Wragge Labs bunker have been churning out all sorts of stuff in the last few months, and I’m way behind in my attempts to document their activities. So this is a bit of a catch-up post to try and commit a few pertinent details to the collective memory bank before they are lost forever in the sleep-deprived fog of day-to-day existence.

Identity upgrades

There have been a number of major improvements to Wragge’s Identity Browser. Regular viewers will recall that the Identity Browser is built on top of the People Australia SRU interface. You might not realise, however, that People Australia contains details of many organisations as well as people. We can only be thankful that it wasn’t called Entity Australia.

The first version of my Identity Browser only searched for people, but now all your corporate-entity-identification needs are also met, with only a few minor changes to the interface so-beloved by numerous generations of identity seekers. To be specific, through the wonders of drop-down technology you can choose whether you want to search for a person or an organisation. Or not. You can also just ignore that and search for everything and get back sensible results anyway. It’s your choice. Or not.

Gaze in awe at the power of my dropdown

Ah pattern matching… there are few phrases so redolent of warm summer days, hidden pleasures, and the subtle delights of wildcard characters. The People Australia SRU interface was sadly lacking in the pattern matching department, but this has now been rectified. So now you mix your stems and asterixes with wild abandon. Searching for ‘Curtin, J*’ will now retrieve all those Curtins whose names begin with ‘J’. Amazing isn’t it?

Astonishing too is the fact that the accompanying ‘Identify me!’ bookmarklet continues to function with nary a murmur of protest. There is, however, a little bit of cleverness built-in to enhance your bookmarklet experience. If the text that you highlight has a comma in it, the Identity Browser will conclude that you’re feeding it the name of a person – ie Surname, Firstname – and will treat the Firstname as a stem. So if you highlight ‘Whitlam, G’ and click on the bookmarklet, the Identity Browser will be kick-started into life, searching for everything that matches surname equals ‘Whitlam’ and firstname is like ‘G*’. If there’s no comma – ie firstname secondname – then it heads off to look for either a person whose surname equals ‘secondname’ and whose firstname is like ‘firstname*’, or an organisation whose name includes both ‘firstname’ and ‘secondname’. Got all that?

Basically the idea was to try and provide some sensible defaults so you really don’t have to think about it too much.

I have it in my head to prepare a long and rapturous homage to the wonders of machine tags. With their sly semantic ways and easy-going nature, they offer some exciting possibilities not just for user-generated content, but user-generated meanings and user-generated relationships. But for the full, ripe pleasure of that post you will have to wait another day, for now I shall simply say that as well as RDFa, the Identity Browser provides automagically-generated machine tags.

Where might you use them? Flickr’s a good place to start. Try identifying the subjects and creators of Flickr photos. At the NSW Reference and Information Services Group Seminar the other day I challenged those in attendance to go forth and machine tag. Already more than 100 machine tags have been added to Flickr using my Identity Browser. Expect to hear more about the Great Flickr Machine Tag Challenge soon…

One more thing… try adding ‘.rdf’ on to the end of an identity record – eg http://wraggelabs.com/identities/person/612109.rdf. Just an experiment at the moment…

More machine tag love

One night on Twitter, @lifeasdaddy pointed out that someone had started using fragments of urls from the NLA newspapers site as tags in the Powerhouse Museum’s collection database. In the conversation that ensued with @sebchan and others, I suggested that the PHM could encourage this sort of rich tagging by supporting machine tags, with all their wonderful juicy semantic goodness The guinea pigs got excited as well, and before I knew it, they’d constructed a little Semweb Helper app.

The Semweb Helper comes with its very own custom-tailored bookmarklet. If you find an article on the NLA newspapers site that you’d like to point to, just click on the bookmarklet and marvel as a range of useful machine tags are automagically generated. Then you just pick the appropriate tag, copy and paste et voila – instant semantic gratification.

Try out the Semweb Helper

It’s a very simple little app, and really just a demonstration of how semantic web technologies might be made available to the masses. It was also the first time the guinea pigs had been allowed to play with the Google Apps Engine.

Who am I?

This short catch-up post has become something quite long and rambling. Did I mention that I’m sleep-deprived? Anyway, a recent addition to the Wragge Labs range of lifestyle accessories is ‘Who am I?’ – a simple little game that is something like a cross between hangman and Wheel of Fortune. Choosing a person at random from People Australia and the Australian Dictionary of Biography, ‘Who am I?’ tests your powers of logic, stamina and historical guesstimation.

Your challenge is to figure out the surname of the mystery historical personage. To help you there are a series of clues, such as their birthplace and known associates. With each guess you also see a little bit more of their portrait. But beware! For ten wrong guesses are all that are permitted to any so brave as to enter upon this quest. Not eleven or twelve, but ten and ten only. To ignore this limit is to invite ridicule and disdain – do so at your peril.

Play Who am I?

‘Who am I’ builds upon some work I’ve been doing for the National Museum of Australia – looking at ways of mashing together various types of date-identified data. As part of that project I’ve built a series of APIs and have scraped, pummelled and munged data from a variety of sources.

What’s the point? I wonder this myself sometimes, particularly after I fling such things off into the aethernet and hear naught but a rare retweet. I am, after all, only in it for the glory, oh and the money of course. (Hmmm, I must look again at that business plan.) The point is twofold: first to highlight possibilities for the re-use and remixing of cultural data; second, to play with game-based models for discovery and exploration of cultural resources; and… err… thirdly just to try building something a little different.

Of course, if you like ‘Who am I?’ you will probably also want to try Headline Roulette…

Headline Roulette Reprieve

At the end of our last instalment, the future of Headline Roulette seemed in dire peril. Changes to the National Library of Australia web site threatened its very existence. Did it have a future? Could it survive? And did anybody care?

As we pick up the story oblivion looms. The feared changes are confirmed, but just as all seems lost… is it? Could it be? Yes, an advanced search facility is added to the newspapers site within Trove. Sensing this may be their only opportunity, the guinea pigs leap into action, building a new screen-scraper, saving Headline Roulette from doom, and setting the world upon the path to a safer, happier future.

In short, Headline Roulette will live on… so enjoy.

Handing out some presents

My head is easily turned by flattery and praise. Yes, I really am so shallow and so vain. But this means that if people say nice things to me, I’m inclined to give them presents.

As well as doing exciting things in the web 2.0 realm for the PROV, @asaletourneau leaves nice comments on this blog. So he earned himself a present. It’s not much, but I built a userscript that displays photos from the PROV site in a neat little slideshow (it’s the non-3D javascript version of CoolIris). Install Greasemonkey, get the userscript and try it out (just do a search, then click on the ‘Browse as slideshow’ button’).

PROV transport photos in a pretty slideshow

The State Library of NSW, or more specifically @ellenforsyth, also earned my favour by inviting me to rave on about Linked Data at the afore-mentioned NSW RISG seminar. As a result, I added support for the SLNSW photo collections to my Flickr Context Harvester userscript. Well… it’s the thought that counts, right? Once again – install Greasemonkey, get the userscript and then try it out.

The Flickr Context Harvester in action

And coming up…

Stay tuned for more on the Great Flickr Machine Tag Challenge, screencasts demonstrating my Identity Browser, some playing with relationships, and much much more. But right now the squirming baby on my lap needs a nappy change…

Did I mention that I’m sleep deprived?

Instead of idly waiting for visitors to stumble over their holdings on some lonely information by-way,  archives are starting to push their content out into the bustling metropolis of the social web. They are going where the people are. Photographic collections, in particular, are gaining new lives and new audiences thanks to Flickr.

But that’s only part of the story. Released into the wild, these photos are slowly picking up the habits of the locals. They are making friends, building connections, even speaking with new accents and dialects. Commented, tagged, organised, linked – they are building new contexts for themselves outside of the cloying control of archival descriptive systems.

Unfortunately it seems there is often a chasm between the old lives of the photos, documented in databases and finding aids, and their new post-institutional careers. This is a pity because the new contexts they are gathering can help us both understand and find them. What can we do to overcome this divide? How could finding aids harvest and display the user-generated content that aggregates around collection items living in the outside world?

The good news is that the tools to start doing this already exist – Flickr has a powerful API that makes it easy to extract photo metadata. Time for a bit of experimenting…

The first result is a userscript that displays Flickr comments in a number of collection databases. Just install it and then try it out:

• National Archives of Australia Photosearch – try it!
• State Records NSW Photo Investigator – try it!
• National Archives and Records Administration ARC – try it!

Flickr comments in PhotoSearch

Gory details follow…

So to begin with I thought I’d just harvest comments from Flickr and display them within existing collection interfaces. As before (here and here), Greasemonkey was my tool of choice for hacking finding aids. The plan was to trigger a Greasemonkey script when you arrive at a photo in a collection database, the script would then:

• extract a unique identifier for the photo that could be used to find it in Flickr
• send off a request through the Flickr API to see if the photo was there
• if so, then fire off another request to retrieve any comments
• format the comments and insert them at a suitable point in the DOM of the database page

Easy! Obviously for the script to work there needed to be a way of connecting entries in the database with photos on Flickr. In practice this means that the photos need to be described at item level, and that a unique identifier needs to be used somewhere in the description of the photo both on Flickr and in the collection database.

Any archive that meets these criteria is a candidate for inclusion. Only three pieces of information are necessary:

• the institution’s Flickr id
• an expression to extract the identifier from the database page
• an expression to identify the point on the database page at which the comments should be inserted

The expressions could use XPath or regular expressions – whatever it takes to find the desired elements. I’m using JQuery, so that makes selecting elements a lot easier. For example, NARA ARC includes the item identifier in a div with the class ‘arcID’, so I just select that element using JQuery and then use regex matching to pull out the number:

this.identifier = $('.arcID').text().match(/ARC Identifier (\d+)/i)[1];

To start with I’ve included the databases of three institutions:

• the National Archives of Australia’s PhotoSearch database
• State Records of NSW’s Photo Investigator
• the US National Archives and Records Administration’s Archival Research Catalog

This is the code to save the settings for each institution:

<br />
if (document.location.href.match(/naa.gov.au\/scripts\/PhotoSearchItemDetail.asp/i)) {<br />
this.name = 'NAA';<br />
this.identifier = document.location.href.match(/M=0&#038;B=(\d+)/)[1];<br />
this.flickrId = '24849862@N08';<br />
this.position = 'table:last';<br />
} else if (document.location.href.match(/records.nsw.gov.au\/asp\/photosearch\/photo\.asp\?/i)) {<br />
this.name = 'StateRecordsNSW';<br />
this.identifier = document.location.href.match(/photo\.asp\?([\d\w_]+)/i)[1];<br />
this.flickrId = '27331537@N06';<br />
this.position = 'table:first';<br />
} else if (document.location.href.match(/arcweb.archives.gov\/arc\/action\/ShowFullRecord|arcweb.archives.gov\/arc\/action\/ExternalIdSearch/i)) {<br />
this.name = 'NARA';<br />
this.identifier = $('.arcID').text().match(/ARC Identifier (\d+)/i)[1];<br />
this.flickrId = '35740357@N03';<br />
this.position = '.genPad:first';<br />
}<br />

From there it’s just a matter of building the calls to the API using Greasemonkey’s built-in  GM_xmlhttpRequest method. Once the comments are retrieved, they’re given some basic formatting and inserted at the point in the DOM identified by the siteDetails.position property. Once again, JQuery greatly simplifies all the DOM manipulation. If there are no comments then a suitable message is inserted together with a link to the photo in Flickr. Finally some CSS is added to prettify it all a little bit.

You can view the full code on the Userscripts site.

Of course, it would be good to have this sort of stuff happening on the server side. In fact, with a few small modifications, this script could just be dropped into the code of any of the collection databases I’ve used. But in the meantime, Greasemonkey gives us a chance to play around with some of the possibilities – to start thinking about what finding aids might be like.

So what’s next? I’d like to do some playing around with tags and locations, perhaps using them to suggest related photos. I’ve also just realised that Flickr machine tags allow semantic markup… hmmm…

If you have any suggestions for databases to add to this script – let me know!

Links mentioned on podcast:
PublicDomainReprints.org
Flickr Commons
MacBook Air
iTunes U
BigThink.com
Berkeley’s YouTube Channel
Google Reader Sharing
ReadBurner
Yahoo Pipes
FeedJournal

Runtime: 51:15
Download the .mp3