Other things on this site...

MCLD
music
Evolutionary sound
Listen to Flat Four Internet Radio
Learn about
The Molecules of HIV
MCLD
software
Make Oddmusic!
Make oddmusic!
[Blog archives] [Categories]

Parakeets in Britain, found in GBIF

I had a great time at the Biodiversity_next conference, meeting a lot of people involved in Biodiversity informatics. (I was part of an "AI" session discussing the state of the art in deep learning and bioacoustic audio.)

I was glad to get more familiar with the biodiversity information frameworks. GBIF is one worth knowing, an aggregator for worldwide observations of species. It's full of millions and millions of observations. Plants, animals, microbes... expert, amateur, automatic observations - lots of different types of "things spotted in the wild". They use the cutely-named "Darwin core" as a data formatting standard (informatics folks will get the joke!).

Here's my first play at downloading some GBIF data. I downloaded all the data they've got about rose-ringed parakeets in Britain - the bright green parrots that are quite a new arrival in Britain, an invasive species which we can see in many city parks now. I plotted the observations per year. I also plotted a second species on the same chart, just to have a baseline comparison. So the parakeets are plotted in green, and the other species (common sandpiper) in yellow:

Many caveats with this data. For a start, each dot represents an "observation" not an "individual" - some of the observations are of a whole flock. I chose to keep it simple, not least because some of the observations list "5000" birds at a time, which may well be true but might swamp the visualisation! Also, some of the co-ordinates are scrambled, for data-privacy reasons - you can see it in the slight grid-like layout of the dots - and some are exact.

Further, I don't think I have any way of normalising for the amount of survey effort, at least for most of the data points. There seems to be a strange spike of parakeet density in 2009 - probably due to some surveying initiative, not to some massive short-term surge in the bird numbers! I think if the numbers really had increased eight-fold and then fallen back again, someone would have said something...

Regarding "survey effort": GBIF does offer ways of indicating survey effort, and also "absences" as well as "presences", but most of the data submissions don't make use of those fields.

The sandpiper data fluctuates too. There's definitely an increase as time goes by, primarily due to the increasing amount of surveys adding to GBIF. That's why I added a comparison species. Even with that, you can clearly see the difference in distribution between the two.

My simple python plotting script is here and the source data are here: parakeet data, sandpiper data.

| science |

State of the Map 2019 - solar conversations

I'm just back from State of the Map 2019, the annual global OpenStreetMap meetup. I gave a talk about the solar mapping project (video here) and had lots of great conversations with people.

Here are some notes from conversations I had about solar PV mapping:

  • I had a good talk with the people from Mapillary. Mapillary have a good collection of street-level imagery, they do automatic detection of objects, and they are already integrated into OSM (e.g. a Mapillary JOSM plugin helps people to use their imagery and even their auto-detections). Note that they're a private company and so I guess they don't share their ML methods or pre-trained networks, though they do explicitly share imagery (and some of their detections) with OSM. They do NOT currently have a tag for solar PV or suchlike. Here's a plan:

    1. They're going to add a "solar panel" tag into their online system. This will be about two weeks from now.
    2. People (us?) will then be able to use their online editor to tag solar PV objects in street-level imagery. This does not directly mean anyone's going to be doing any auto-detection. But it does give us a new and different source of imagery to spot solar PV in.
    3. We'll be able to use the Mapilary plugin (I think) to add those streetside-spotted items to the map.
    4. If we can tag a good diverse set of small-scale solar PV - say, a few hundred of them, in different countries - then we will be in a position to start chatting to Mapillary about whether they're interested in training some auto-detection for that tag.
  • In the UK we've mapped solar farms with the power=plant tag, which is good because it makes them clearly different from the individual solar panels tagged as power=generator. However, around the world this isn't the case. Lots of people have mapped solar farms with single large power=generator tags. This is not necessarily terrible because we can identify large objects when we postprocess. It's problematic if people are using single nodes to mark solar farms though, since it's hard to tell the difference between a 10 kW and a 100MW installation without more detail. I hope we can convince the wider community that our way of tagging (which comes directly from the established power tagging) is the way to go.

    • Although the editing tools have a preset for solar panels, they don't have one for solar farms / solar parks. We should try to propose a preset for JOSM and iD. It would fill in the basic tags, and offer a textbox for entering the capacity. Also it'd be good to ofer a text box for the start date, to gather that if known.
    • The default map rendering on openstreetmap.org doesn't render anything for the power=plant tag. This may be acting as an incentive for people to stick with power=generator, because of the visibility of the results. (Some people add landuse=industrial to solar farms, which is no problem for us; I don't know if they're doing it because it renders quite nicely.) We should ask the maintainers of the openstreetmap-carto rendering style if they're willing to render solar farm areas, perhaps just the standard industrial colour.
  • I met Julien Minet who had created a highly detailed service for the city of Brussels. The city commissioned a LiDAR survey of all the buildings, giving them a very detailed 3D model. The developers used this model to predict the solar energy yield for every single roof part (can be multiple on each building!), and made this into a very nice app for people to predict their personal projected benefit of installing solar PV. This is the kind of detail that only a well-organised city can feasibly manage... It's a high-res version of what "open solar map" did for France, which is a more scalable example.

  • How to vectorise blobs from pixelwise detections... it seems it's fairly common knowledge. QGIS at least has a function for it ("polygonize") which offers at least a starting point, though it's very simple-minded (it traces all the edges of pixels - you need to run a "simplify geometries" afterwards). Facebook's new "rapid" ML-assisted editing tool proposes building shapes for the user to add, so they are doing the process too somehow. (I'm pretty confident they're doing that as postprocessing, not some fancy ML that directly spits out geometries.) I don't know which are the current leading methods for this.

  • I like the look of Facebook's approach to machine-augmented editing. They made a modified version of the iD editor (and named it rapiD), which offers machine-proposed areas for the user to confirm-and-edit one by one. I wasn't able to attend their demo though so I don't have more details e.g. whether it's stable, whether we could adapt it.
  • There's lots of latent interest in trying machine learning detectors on aerial imagery - I saw a couple of talks about doing this for other purposes. Largely using methods we'd recognise from other work (i.e. convnets). I saw a talk from IBM using CycleGAN which seemed to me a very odd choice of method, I'd not recommend that! On the other hand, I started to wonder whether we should be organsing a small-scale-solar-PV-detection data challenge. (We don't need to worry about detecting large-scale solar, there's existing work on this already.) We've got sufficient annotations for the UK now to make ML training sets - I guess the key question is whether we can get good enough imagery, and whether it's something that'd be of benefit. We could use aerial or even streetside imagery. As well as Mapillary I chatted to Maxar (aerial imagery providers) - the Maxar fellow mentioned a US company/charity called something like CrowdSpark (I THINK - or maybe it was a different company) that has organised data challenges related to development/aid. If we did put together a data challenge, what approach would be most likely to lead to detections that (a) could help us complete the UK, (b) would help us to scale globally? I don't know if this is worth pursuing though - can perhaps simply rely on existing object detectors rather than assuming it needs novel effort - TBC.

  • I caught up with Jerry (SK53) who was a key player in the OSM UK solar mapping. Didn't manage to catch up with Jez Nicholson - somehow we missed each other!

Overall, there was lots of enthusiasm for the idea of mapping all the solar PV, and I'm really glad to have been at SotM to talk about it.

| openstreetmap |

Solar panels in the UK - 100,000 spotted!

The OpenStreetMap UK community has come together for a 3-month "quarterly project" to find all the solar panels in the UK. And the results so far... wow!

-> We've just reached over 100,000 standalone solar PV installations mapped in the UK!

-> Plus we also have almost 600 solar farms mapped, representing almost 5 gigawatts!

To give you an idea of completeness: the UK government believes there are about 1 million solar panels in the UK, and official solar farm listings have about 8 GW. What we've mapped so far is way more than I thought we'd manage, so I really deeply want to thank every person who contributed, large or small. We can use these data to pilot CO2-saving initiatives - and so can anyone else. Open data for the win!

Here are a couple of simple plots from me, to show where the items are in the UK. (The data I've used are from yesterday, which is why my plot says not-quite-100,000 items.) I've plotted two different types of item: (1) "standalone" solar panels (i.e. ones that are not inside solar farms); (2) solar farms. You'll notice the distributions are really different:

Some of the darkest blobs on the map are evidence of focussed effort. I know that Jerry (SK53) has been mapping around Nottingham, Cornwall and other areas, so may be the source of some of these blobs. We've shown that with some local effort, and a bit of scanning through imagery, a lot of this can be done.

You'll notice that many of the solar farms are in the southern part of the UK, at least according to our data. That's not unexpected!

For more detailed data breakdowns, you can peruse Gregory Williams' solar mapping analysis site.

We have more time to go until the end of the quarter... and we'll be able to use the data for sustainable energy projects whenever you contribute - so please do join in!

| openstreetmap |

IBAC 2019 Brighton - bioacoustics research

The 2019 International Biocaoustics Congress (IBAC) was its fiftieth year! And it was a very stimulating conference, held in Brighton and organised by David Reby and a lovely team of co-organisers. Shout out to Eilean Reby who designed a really neat visual identity for the conference, with T-shirt and bags designed for everyone - we had a choice of designs featuring bat, bird, insect, fox...

It was my third IBAC (after IBAC 2015 in Germany and IBAC 2017 in India) and I was very happy to chair a session on advanced computational methods, and to see some great work in there. In this post I'll make notes of other things I saw.

Our QMUL bird group presented their recent work. Lies Zandberg spoke about her great work with Rob Lachlan, as part of our project to understand sound similarity from a birds' point of view. Mahalia Frank presented her studies (her PhD with David Clayton) on the neuroanatomy that might explain what zebra finches can hear while still in the egg.

The most shocking bird news, for me, came from the superb lyrebird. Now, the superb lyrebird is a famously showy Australian songbird - the male does this bizarre-looking dance on top of a little mound he's built. The shocking revelation was in Anastasia Dalziell's talk about her studies on the lyrebird's mimicking song. She showed us evidence that the song incorporates sounds from other bird species' mobbing calls (i.e. what groups of birds do when there are predators about), and further, that this mimicry might have evolved to con the female lyre bird into staying around for some copulation when she may be on the verge of wandering away. It's one possible way that complex mimcry might arise through evolution (antagonistic co-evolution) ...and it's very odd behaviour.

Also making a splash was ... RoboFinch! A very eyecatching piece of work presented by members of Katharina Riebel's lab, as part of a project called "Seeing Voices". Ralph Simon presented about the technical development, and Judith Varkevisser on her vocal learning studies. The idea is to explore how the visual modality affects vocal learning. They tried various things: young birds housed with the tutor, or watching the tutor from the next chamber, or hearing him only; or showing videos of the tutor to the young birds; then a "hologram" (a Pepper's ghost) bird... Then they went as far as building lifesize animatronic zebra finches, with robotic heads/beaks and colours painted to match the exact colour profile of real birds. It seems the young birds really do like to get along with RoboFinch, much more than the other methods, or the traditional audio-only "tape" tutoring.

We all like a clever and informative data visualisation technique. A few years ago it was Michael Towsey's "false colour spectrograms" I was excited about - and we saw some use of those at IBAC 2019. A new visualisation was Julia Hyland Bruno's work (part of her PhD with Ofer Tchernikovsky) - surprisingly simple, she renders many different renditions of a zebra finch song as a colourful image in which each row is a colour-mapped time series for one rendition. With a bit of judicious alignment, the effect is to give an immediate visual impression of the repeatability and the structure in the song. A paper showing the technique is here.

I continue to be interested in rhythm and timing in animal and human sounds, and IBAC is a good place to catch some research on this.

  • Michelle Spierings presented a very intriguing new method she has been developing with Tecumseh Fitch: a "palimpsest" scramble of words on-screen, while the sound of a repetitive kick drum plays. The question: does the kick drum help you to entrain and spot the words appearing on the beat? The answer: yes. There are some other neat things you can think of trying with this method.
  • Isabelle Charrier showed acoustic recognition of northern elephant seal mates used pulse rate (and spectral shape, spectral centroid) were super stable carriers of individual ID. --- Intriguingly, Juliette Linossier (separately) presented her work on indivudal ID in the sound of northern elephant seal pups: the mothers can recognise their pups' sounds, but those pups' sounds are weird-sounding creamy grunts, and it's very hard for us listeners to guess how the individual identity might be represented in there.
  • Florencia Noriega showed her quantitative method for comparing rhythm patterns in animal sound sequences. It's based on the well-known inter-onset intervals (IOIs) and she has applied it to frogs, parrots and zebra finches, showing that it makes an apparently discriminative compact representation.
  • Nora Carlson showed the very interesting setup she has at the Max Planck Institute of Animal Behaviour - a group of free-flying birds in a large barn, 3D motion-tracked using a Vicon camera system. She's running a study on vocal communication networks - which is of interest to me given that I've analysed smaller vocal networks of birds before. In fact in the paper I just linked to, there's a reanalysis of some of the work Manfred Gahr presented later in the same session - they have some great work studying bird vocal communication via tiny little backpacks that birds carry around. From the same group, Susanne Hoffmann presented their study of duetting weaver birds in the wild - amazing that they're live-recording the audio as well as neural activity of pairs of birds out in the wild.

Good to catch up on large-scale and ecoacoustic monitoring too.

  • Sarab Sethi and Becky Heath presented their work on large amounts of acoustic data gathered in Borneo. They used 3G-connected Raspberry-pi based devices. I think Sarab's method of using the off-the-shelf Google "AudioSet features" as an audio representation has many appealing qualities: unsupervised, reproducible, and clearly representing ecoacoustic data in at least a partly-semantically-disengangled fashion. There are some definite residual questions: someone pointed out that since AudioSet is based on human-oriented Youtube audio, it's unlikely to represent high frequency events usefully, and clearly doesn't represent ultransonics at all. It would be a good thing if, as a community, we created a large "EcoAudioSet" and a fixed feature representation derived from it.
  • Plenty of projects using AudioMoth as their audio gathering tool. One nice example is Tomas Honaiser Rostirolla's project to document Brazilian ecoacoustics - they made 20274 manual annotations (3988 of which at species level) from 6048 recordings, and openly published I believe. Great.
  • Wesley Webb got a lot of attention, not only for his prize-winning presentation of his study on bellbird song in New Zealand, but also for introducing "Koe", a web-based an open-source tool intended to speed up the manual process of labelling large amounts of bioacoustic audio data.
  • Paul Roe gave us an overview of their "acoustic observatory" in Australia, and in particular the institutional and collaborative efforts needed to create and maintain such a large monitoring project. For example, the use of piggy-backing on other researchers' projects ("heading out in to the bush? could you check this recorder for us while you're there?")

As usual for IBAC, I learnt some amazing things about bioacoustics outside the realm of birds. Some great bat work represented, e.g. from Sonja Vernes' group (I remember IBAC 2015 when I think she might have been literally the only person talking about bats).

One little observation: both Julie Oswald and Jack Fearey had used a neural net called "ARTwarp" to cluster the vocalisations in their whale data, to try and understand their repertoires. I don't know this method, but it seems to be an unsupervised clustering method incorporating time-warping - might be of interest.

And even more, I continue to learn from the weird world out there beyond the vertebrate animals. Amazing little insects that nibble holes in leaves, for example, so that they can create little acoustic cavities to broadcast well. The most out-there bit of acoustic communication was as follows... a type of ants drilling holes in acacia tree thorns, which then create a whistling signal in the wind, to scare away cattle. Even better, the researchers are currently tested this hypothesis by drilling thousands of little holes in trees in Africa, to see if the reduced whistling affects the cattle behaviour. (This was presented by Kathrin Krausa on behalf of Felix Hager.)

Oh one more thing -- the vocal imitation competition had some fantastic results. As the final event of the entire conference, we listened to a deluge, a menagerie of inexplicable animal sounds that had been imitated by IBAC attendees themselves. This could become a fixture in the calendar...

| science |

Should we blame Boris, Theresa, or David?

It looks like the bigger consequences of the Brexit vote are about to hit. Everyone thought "no deal" was a laughable extreme back in 2016, and now our government seems to be sailing deliberately towards it.

Do we blame David Cameron, who naively called an ill-prepared vote? Theresa May who dogmatically stuck to extreme "red lines" in the negotiations, failed to get a majority in a General Election, failed to get the agreement voted through, and yet dogmatically refused to consider compromise positions, again and again? Or do we blame incompetent cartoon character Boris Johnson - whose main achievement as London Mayor was the foolish "garden bridge" plan that went nowhere after wasting tons of our money - now arrogantly pushing us to no-deal despite the danger for ordinary people, as well as the democratic deficit?

It's crucial to remember that none of these people is at the root of all this. The Conservative Party as a whole should carry the blame. (Or the credit, if no-deal is a success - sure, why not.) They are currently pushing a heck of a lot of effort into trying to blame the EU for any no-deal Brexit: pretending the EU is refusing to negotiate, when in fact the EU is refusing to reopen negotiations it's just spent two years on, or at least refusing to reopen them unless the UK government proposes a way forwards. The Conservative party have got us into this mess - not just the British public! There were many different routes the government could have taken from 2016 onwards, and this embarrassing mismanagement comes from the Conservative party, again and again. Their MPs and leaders, the government ministers, the membership. We can consider blaming Corbyn for not being an EU cheerleader, but frankly, his subdued triangulating is a minor footnote in this shambles of bad tactics.

I might be tempted to go deeper and say our First-Past-The-Post (FPTP) voting system is on the hook, since that's the reason that the Conservative Party (and Labour) sticks together in the shape it does, and why they feel the need to pander to extremists. But that might be a little too indirect.

We're already suffering the consequences of the Brexit vote. (Personally: I've lost multiple colleagues who have gone overseas, lost opportunities, etc. Collectively, we've lost a lot of influence, and we've wasted a heck of a lot of time we could have been spending fixing the climate crisis.) I'm particularly concerned about what will happen next. In particular, a perverse incentive of a crisis such as no-deal: it gives the sitting government a rare opportunity to ram through emergency legislation which might reshape the British settlement more radically than is normally achievable. The Conservative Party has already shown simple-minded cruelty in voluntarily imposing a cost-slashing "austerity" agenda on the UK for a decade - an agenda wihch may have led to over a hundred thousand excess deaths. They have demonstrated that they don't worry about whether poorer people suffer the side-effects of their big ideas. What more would they like to do?

We see some hints of changes that Brexit might enable: today the food industry claimed it's going to need exemption from the laws of fair competition in order to keep everybody fed. That argument wouldn't get anywhere near the table in ordinary times.

I'm opposed to Brexit but I certainly think naively "cancelling" it would be as harmful as going forward. The right way to proceed, clearly, is to take more time to find a true compromise outcome (which may well turn out to be a soft-ish Brexit, although no-one on any side wants to admit that). The rush now is for selfish reasons. We don't yet have a plan. - And, for our current government, perhaps it's more than just a necessary evil, perhaps not-having-a-plan is an opportunity they are looking forward to?

| Politics |

The best alcohol-free beers in Europe

We've sampled LOTS of alcohol-free beer in the past year. Why? Well - you might not believe me about this - some of it's getting really good. And it's great to be able to have a lovely beer even if you don't want to be woozy afterwards.

So WE HAVE DONE THE RESEARCH and YOU can benefit from it! We've drunk beers good and bad from all over Europe. Here are six fabulous alcohol-free beers that are truly excellent and you should DRINK THEM!

bottle photo1. Mariestads "Sommarlager" (Sweden, 0.5%): a delightful lager, perfectly balanced, fizzy yet with creaminess/butteriness. I'm amazed that a lager (rather than an IPA) is in here in the top spots but there you go. It's refreshing and beery and delightful.
bottle photo2. van de Streek "Playground IPA" (Netherlands, 0.5%): This IPA is phenomenal, with the full complex and vibrant flavour you want from a modern IPA. I don't know how they did it. Out of all the low-alc IPAs, this one is the one that can compete with normal ones.
bottle photo3. Värmdö "Ingarö Eko IPA" (Sweden, 0.5%): delightful unsweet, slightly sour almost kombucha-y, yet juicy and no bad notes. I liked this one for being a little bit different.
bottle photo4. Big Drop "Pale Ale" (UK, 0.5%): Another IPA that passes the test. I've no notes on this one, and only a fuzzy paparazzi photo (must have been a good night huh?) so you'll just have to try it.
bottle photo5. Mikeller "Drink'in the sun" (Denmark/Belgium, 0.3%): A good IPA (allegedly a wheat pale) with summery juicy grapefruit.
6. Clausthaler "Original 0.5%" (Germany, 0.5%): this is in the classic German style, and was the first time I drank an alcohol-free beer and felt refreshed, without that curious slightly-wrong flavour that's in the mainstream alcohol-free beers. Since I first tried it, I discovered the other ones, but this one is still great, and notable for getting the balance of flavours just right to make a quality German beer despite the constraint.

Others which are ALSO recommended:

  • Mariestads "Alkoholfri" (Sweden, 0.5%): good reliable beer.
  • Sigtuna "NAPA" (Sweden, 0.5%): good IPA with a hint of Nanny State.
  • Adnams "Ghost Ship" alcohol-free IPA (UK, 0.5%): pretty convincing! A little bit drying imho.
  • Brewdog "Nanny State" (UK, 0.5%): Brewdog's well-known low-alc, with a very nice complex flavour, a malty pale ale.
  • Sainsburys "Low alcohol Czech lager" (Czechia, 0.5%): Surprisingly not bad for a supermarket lager. Pilsner-style.

Plus there are plenty I don't recommend, of course :) but I won't bore you with them. Some more of my tasting notes are in this twitter thread.

While writing this, I found a nice list of top 70 low-alcohol beers from "Steady Drinker". However... it looks like he hasn't been to Sweden! He hasn't reviewed Mariestads, Värmdö or Sigtuna, which is why he's missed some of my top ones. In Sweden they have quite hefty taxes on alcohol, which may be why it's become a great place for innovative low-alcohol beers.

| food |

Holiday by train - from Britain to Sweden

Our holiday this year was great "grounded travel" - we went from the UK to Sweden, going all the way by train! We stopped in multiple cities on the way, in Germany and Denmark as well as Sweden.

I want to tell you how we did it. But before all that …

| eco |

Running Mopidy as a service with PulseAudio on Ubuntu - gotcha

I've been enjoying using Mopidy and ncmpcpp as my music player.

On Ubuntu though I encountered a persistent problem:

  1. I had set Mopidy up to run as a service, which seemed like a good idea - it's not just some application I want to start from time-to-time, it's the main music …
| linux |

Tool needed: simple solar spotter

The OSM UK community is great, but it's hard to guarantee we can do the detective work to spot all 800,000 solar PV installations. There's a "long tail" of solar panels tucked away down side-streets. It's very much a needle in a haystack, and we would benefit from as …

| openstreetmap |

The H-index can not be relied on for recruitment

The H-index is one of my favourite publication statistics. It's really simple to define: a person's H-index is the biggest number H of publications which have been cited H times each. It's robust to outliers: if you've a million publications with no citations, or one publication with a million citations …

| science |

social