flickr promoting the commons / creative commons

flickr is promoting photos in what it calls "The Commons", but only to logged in users. Normal users (who can't comment / tag the photos in the commons anyway) don't get an obvious link to them (except via about a billion third party sites such as blogs and google search). The page also shows how the national library's choice not to include text in their logo has come up trumps.

The confusingly similarly named "The Commons" and "Creative Commons" parts of the website apparently don't reference each other. Odd.

Excellent stuff from New Zealand Geographic Board Ngā Pou Taunaha o Aotearoa

A while ago, motivated by the need for an authoritative list of New Zealand place names for our with at the NZETC, I criticised the NZGB fairly roundly.
While they haven't produced what I/we want/need, in the last couple of months they've made huge progress in an unambiguously right direction.
Their primary work is the New Zealand Gazetteer of Official Geographic Names, a list of all official place names in New Zealand. It uses have a peculiar definition of "official" (= mentioned in legislation or a Treaty of Waitangi settlement), they have very few names of inhabited places (and no linking with the much larger ones maintained by official bodies such as the police and fire service), They have no elevation data for mountains and pass (which are defined by their height) and they define some things as points when they appear to be areas (such as Arthur Pass National Park), but it's much better than the New Zealand Place Names Database since:

1. It has a statutory reference for every place, given the source of the officialness of the name
2. It fully support Macrons
3. It has a machine readable-list of DoC administered lands --- I can imagine this being used for all sorts of interesting things, getting people out in other scenic and marine reserves.

NZGB sent around an email in which they explicitly addressed some of the points I'd earlier raised (I'm sure I wasn't the only one):

It should be noted that some of the naming practices of the past will have to be lived with, despite inconsistencies. Moving forward, the rules of nomenclature followed by the NZGB are designed to promote standardisation, consistency, and non-ambiguity. The modern format for dual names is '<Maori name> / <non-Maori name', which the NZGB has applied for the past 10 years, though Treaty settlement dual names sometimes deviate from this convention, because the decision is ultimately made by the Minister for Treaty of Waitangi Negotiations. Older forms of dual names, with brackets, will remain depicted as such until changed through the statutory processes of the NZGB Act 2008. These are not generally regarded as alternative names.
Macrons in Maori names have posed problems for electronic databases. Nevertheless they are part of the orthography, recommended by the Maori Language Commission, and the Board endorses their use. The Gazetteer will include macrons where they are formalised as part of the official name. When Section 32 of the new Act comes into force, official documents will be required to show official names, and these will need to include macrons where they have been included as part of the official name (unless the proviso is used). A list of those official names which have macrons is at http://www.linz.govt.nz/placenames/researching-place-names/macrons/index.aspx . LINZ's Customer Services has some solutions for showing macrons in LINZ's own databases and on published maps and charts, and is currently investigating how bulk data extracts might include information about macrons, for the customer's benefit.

Despite the name, it isn't clear in my mind exactly what's official and what isn't. Is the content of the "coordinates" column official? For railway lines this is a reference to the description, which in the cases of railways is usually of the form "From X to Y", where X and Y are place names, frequently place names that aren't on the list, so are thus presumably not official. Unless I'm going blind there is also no indication of accuracy on the physical measurements.

fuzzziness

I've been using topic maps in my day job, so I decided to try out http://www.fuzzzy.com/, a social bookmark engine that uses an underlying topic map engine.
I tried to approach fuzzzy with an open mind, but the increasingly stumbling on really annoying (mis-)features.

1. This is the first bookmark engine I've ever used hat doesn't let users migrate their bookmarks with them. This is perhaps the biggest single feature fuzzzy could add to attract new users, since it seems that most people who're likely to use a bookmark engine have already played with another one long enough to have dozens or hundreds of bookmarks they'd like to bring with them. I know this is non-ideal from the point of view of the social bookmark engine they're migrating too, since it makes it hard to do things completely differently, but users have baggage.
2. While it'd possible to vote up or vote down just about everything (bookmarks, tags, bookmark-tags, users, etc), very little is actually done with these votes. If I've viewed a bookmark once and voted it down, why is it added to my "most used Bookmarks"? Surely if I've indicated I don't like it the bookmark should be hidden from me, not advertised to me.
3. For all the topic map goodness on the site, there is no obvious way to link from the fuzzzy topic map to other topic maps.
4. There doesn't seem to be much in the way of interfacing with other semantic web standards (i.e. RDF).
   
5. The help isn't. Admittedly this may be partly because many of the key participants have English as a second language.
6. There's a spam problem. But then everywhere has a spam problem.
7. It's not obvious that I can export my bookmarks out of fuzzzy in a form that any other bookmark engine understands.

These (mis-)features are a pity, because at NZETC we use topic maps for authority (in the librarianship sense), and it would be great to have a compatible third party that could be used for non-authoritative stuff and which would just work seamlessly.

Place name inconsistencies

I've been looking at the "Dataset of New Zealand Geographic Place Names" from LINZ. This appears to be as close as New Zealand comes to an Official list of place names. I've been looking because it would be great to use as an authority in the NZETC.

Coming to the data I was aware of a number of issues:

1. Unlike most geographical data users, I'm primarily interested in the names rather than the relative positions
2. New Zealand is currently going through an extended period of renaming of geographic features to their original Māori names
3. The names in the dataset are primarily map labels and are subject to cartographic licence
   

What I didn't expect was the insanity in the names. I know that there are some good historical reasons for this insanity, but that doesn't make it any less insane.

1. Names can differ only by punctuation. There is a "No. 1 Creek" and a "No 1 Creek".
2. Names can differ only by presentation. There is a "Crook Burn or 8 Mile Creek", an "Eight Mile Creek or Boundary Creek" and an "Eight Mile Creek" (each in a different province).
3. There is no consistent presentation of alternative names. There is "Saddle (Mangaawai) Bivouac", "Te Towaka Bay (Burnside Bay)", "Queen Charlotte Sound (Totaranui)", "Manawatawhi/Three Kings Islands", "Mount Hauruia/Bald Rock", "Crook Burn or 8 Mile Creek" and "Omere, Janus or Toby Rock"
4. There is no machine-readable source of the Māori place names with macrons, and the human readable version has contains subtle difference to the machine-readable database (which contains no non-ASCII characters). For example "Franz Josef Glacier/Kā Roimata o Hine Hukatere (Glacier)" and "Franz Josef Glacier/Ka Roimata o Hine Hukatere" differ by more than the macrons. There appears to be no information on which are authoritative.

Right now I'm find finding this rather frustrating.

(grammar edit)

Does anyone publish the Dataset of New Zealand Geographic Place Names already in XML form?

I've been playing with the Dataset of New Zealand Geographic Place Names which is a set of CSV files published by Toitū te whenua / Land Information New Zealand (LINZ). The data takes quite a bit of massaging, and I was wondering whether anyone else had already done the work of making acceptable XML out of the data rather than doing all the work myself.

I've attached the script I have so far, but it's not perfect. In particular:

1. It doesn't include place names with Macrons
   
2. It makes lots of ASCII-type assumptions
   
3. Many of the element names are poorly named and map non-obviously to fields in the CSV files.
4. The script isn't very generic and does little or no checking

Anyway, here's he script, hopefully it's successfully escaped. The basics are that it creates an sqlite database and streams the CSV files into it direct from the zip (which it expects to have been downloaded into the current directory). It then streams each point out using awk to transform it to XML.

#!/bin/bash
# script to import data from
# http://www.linz.govt.nz/placenames/search/place-names-dataset-download/index.aspx
# into an XML file.
# this script licensed under the GPL/BSD/Apache 2 licences

echo \(re\)creating the database, expect DROP errors the first time you run this 
sqlite nzgeonames.db << EOF
DROP TABLE name;
CREATE TABLE  name (id, name, east, north, pdescription, district, sheet, lat, long);

DROP TABLE district;
CREATE TABLE  district (district, description);


DROP TABLE pdescription;
CREATE TABLE  pdescription (pdescription, short, description);


DROP TABLE sheet;
CREATE TABLE  sheet (edition, map, sheet);

VACUUM;
EOF

echo importing the names
unzip -p nznames_6Aug08.zip namedata.txt  | sed 's/\r//'  |  sed 's/`/","/g' | awk -F^ '{print "INSERT INTO name VALUES (\"" $0 "\");"}' | sqlite nzgeonames.db 

echo importing the districts
unzip -p nznames_6Aug08.zip landdist.txt  | sed 's/\r//'  |  sed 's/`/","/g' | awk -F^ '{print "INSERT INTO district VALUES (\"" $0 "\");"}' | sqlite nzgeonames.db

echo importing the point descriptions \(expect two lines of errors\) 
unzip -p nznames_6Aug08.zip pointdes.txt  | sed 's/\r//'  |  sed 's/`/","/g' | awk -F^ '{print "INSERT INTO pdescription VALUES (\"" $0 "\");"}' | sed 's/:/","/' | sqlite nzgeonames.db

echo importing the sheet names
unzip -p nznames_6Aug08.zip sheetnam.txt  | sed 's/\r//'  |  sed 's/`/","/g' | awk -F^ '{print "INSERT INTO sheet VALUES (\"" $0 "\");"}' | sqlite nzgeonames.db 


# pick up the ugly duckling
sqlite nzgeonames << EOF
 INSERT INTO pdescription VALUES ("MRFM","MARINE ROCK FORMATION","Marine Rock Formation");
EOF

echo exporting points as xml
echo "<document source=\"Sourced from Land Information New Zealand, [date]. Crown copyright reserved.\">" > nzgeonames.xml
sqlite nzgeonames.db "SELECT name.id, name.name, name.east, name.north, name.pdescription, name.district, name.sheet, name.lat, name.long, district.description, pdescription.short, pdescription.description AS descriptionA, sheet.edition, sheet.map FROM name, district, pdescription, sheet WHERE name.district = district.district AND name.pdescription = pdescription.pdescription AND name.sheet = sheet.sheet;" | awk -F\| '{print "<point><id>" $1 "</id><name>" $2 "</name><east>" $3 "</east><north>" $4 "</north><pdescription>" $5 "</pdescription><district>" $6 "</district><sheet>" $7 "</sheet><lat>" $8 "</lat><long>" $9 "</long><description>" $10 "</description><short>" $11 "</short><descriptionA>" $12 "</descriptionA> <edition>" $13 "</edition> <map>" $14 "</map>  </point>"}' | sed 's/&/&amp;/' >> nzgeonames.xml 
echo "</document>" >> nzgeonames.xml

echo formatting the points nicely
xmllint --format nzgeonames.xml  > nzgeonames-formatted.xml

Library of Congress flickr experiment

Native troops, Cuba (LOC)
Originally uploaded by The Library of Congress

While processing the photos from my parent's ruby wedding anniversary, I ran into the Library of Congress's flickr experiment.

I probably shouldn't have been, but I was astounded. It looks the the bastion of old-school cataloguing is coming to bathe in the fountain of social tagging.

This is part of a larger effort described at http://www.flickr.com/commons/

Saxon joy!

I've just moved to saxon from libxml for some XSLT stuff I'm doing, and I'm really loving it.

Not only does saxon run take much less memory, it also speaks XSLT 2.0.