In their recent paper on “content growth and attention contagion” (free preprint), researchers Kai Zhu, Dylan Walker and Lev Muchnik use Wiki Education’s Wikipedia Student Program as a natural experiment to study the effects of a sudden “shock” of new editing activity. They look at the set of about 3,300 articles that were substantially expanded by student editors during the Fall 2016 term, compared to a control group of similarly developed articles that weren’t worked on by any of our classes. The main idea is to look at what happens after the classes have ended and the students have (largely) stopped editing.

One major effect is an increase in pageviews — up 12%, on average, compared to the control group, and the more an article expanded, the larger the impact on pageviews. Not only that, but there’s a “spillover”, with increased pageviews for the articles linked to from the students’ articles — especially for newly added links. Along with more pageviews, these students’ articles got more edits from more unique editors after the class ended. As with the pageview effect, the more an article was expanded during the Fall 2016 term, the more edits and editors it was likely to attract afterwards. This increased attention for the students’ articles represents a combination of more traffic coming from related articles that link to them as well as more direct traffic from search results.

Their findings underscore the impact of our work, and reinforce a few ideas that are at the heart of what we do. First, it shows the impact and scale of our Wikipedia Student Program: student editors are doing so much to expand Wikipedia’s content gaps, especially in academic content areas, that it’s possible to study their activity and find statistically significant effects. (In an average month, our program participants represent nearly 20% of English Wikipedia’s new active editors.)

It also shows that the work students are doing isn’t just an “academic exercise”; it has an audience, and it matters enough that people are both reading it and building on it. Because of the way articles lead readers to explore the network of related articles, this “attention contagion” for substantially improved articles spills over to related articles, and this suggests that a particularly effective strategy is to focus on clusters of underdeveloped but important articles. That happens naturally in many of our Student Program courses, where the instructor guides their students to work on underdeveloped articles related to the course content. We’re also doubling down on this strategy for our Wiki Scholars & Scientists courses, where a cohort of experts comes together to learn how to apply their knowledge to a particular topic area — such as our recent courses focused on the COVID-19 pandemic.

Research like this is an important part of the Wikimedia ecosystem. Not only does this help quantify the impact of Wiki Education’s work, but it also offers insight for other Wikimedia program leaders interested in focusing editors on a cluster of underdeveloped but important articles. Wiki Education is always interested in independent researchers using our programs as a focus of academic study; for information on data available, reach out to us through our contact us page.

By Andrew Otto, Staff Site Reliability Engineer

In the previous post, we talked about why Wikimedia chose JSONSchema instead of Avro for our Event Data Platform. This post will discuss the conventions we adopted and the tooling we built to support an Event Data Platform using JSON and JSONSchema.

For Avro serialized streaming event data, the event’s Avro schema is needed during both production and consumption of the event. For JSON events, the schema is not strictly needed to produce or consume the data. JSON records include minimal ‘schema’ information, including the structure of the record and the field names. Field type information is what is missing.

If we are using JSON, why do we need the schemas at all? Producers and consumers don’t need it. What does?

There are two answers to this question.

1. We want to validate that all incoming events of a specific type conform to a schema at produce time, so that consumers can rely on it being consistent.
2. Schemas are useful for data integration, e.g. ingesting event data into an RDBMS or elsewhere.

To do either of these, we need to be able to find the schema of an event.

Confluent does this with opaque schema IDs which are used to look up schemas in their remote Schema Registry service. This means that each schema gets assigned its own unique integer ID inside of the Schema Registry (just like a unique ID in a database). This ID carries no meaning outside of a particular Schema Registry installation. 

But we’d like the ability to lookup a schema without access to a centralized registry. Wikimedia does this for JSONSChemas with versioned URIs.

Schema URIs

JSONSchema documents themselves have their own ‘meta schemas’. The meta schema of a JSONSchema document is identified using the $schema keyword, which usually is set to the official URL of a JSONSchema version, e.g. ‘http://json-schema.org/draft-07/schema#’. JSONSchema validator implementations use this to know which version of JSONSchema to use when interpreting a JSONSchema document. 

So $schema is used to identify and lookup the meta schema of a JSONSchema document.

This kind of lookup is exactly what we need for our JSON event data: we want to lookup the schema of the JSON event document. We add the $schema keyword in our event data itself and set it to the URI of the event’s JSONSchema. Here’s an example event:

{
  "$schema": "http://schema.example.org/coolsoftware   /user_create.json",
  "datafield1": "value1" 
}

With the $schema present in every event and set to a URI pointing at the event’s schema, any user of the event can get its schema at will. Wikimedia’s EventGate HTTP -> Kafka proxy uses $schema to validate events before they are produced to Kafka. We use the same field to lookup the JSONSchema and convert it to a Hive or Spark schema during ingestion into Hadoop. We’ll come back to use cases like this later.

Decentralized Schema Repositories

In the first post in this series, I described how Wikimedia code and data should be decentralized. But the example event above had a fully qualified URL for its $schema, which would require anyone who wanted to lookup that event’s schema to do so from a centralized address. This also couples the schema lookup to that remote website. If the schema website goes down, schema lookup will fail.

Git is great at decentralization, so we keep all schemas in git repositories. This means that users can find schemas wherever the schema repository is cloned: in your local file system, somewhere in a distributed file system, or somewhere at a remote web address. Since these locations will all clone the same git repository, we can assume that they have the same relative directory hierarchy and choose to only use relative $schema URIs in event data.

Our example event would instead look like this:

{
  "$schema": "/coolsoftware/user_create.json", 
  "datafield1": "value1" 
}

We’ve removed the reference to a centralized web address…but what now? This surely isn’t enough to look up a schema.

It isn’t! We will need to prefix the $schema URI with a base address or path. If we want to read the schema from a local clone of our schema repository, we can prefix the relative $schema URI with a file:// path like file:///path/to/schema_repo/coolsoftware/user-create.json. If the schema is hosted at a remote HTTP address, we can prefix it with that domain like http://schema.example.org/coolsoftware/user-create.json. 

This is especially useful in development environments while developers create and change schemas and producer code. Their environments can configure a local schema base path to their clone of the schema repository. This also helps to decouple services in production. A critical service that needs to lookup schemas can have a local copy of the schema repository, whereas a less critical service that might want to do more dynamic schema lookups can use a remote web address to the schema repository.

Versioned Schemas

Code changes over time and so does data. As the data changes, so must the schemas. In an event driven architecture, event data is immutable and captures the historical record of all changes. We need a way to modify event schemas so that all historical event data of the same schema lineage will validate with the latest version of that schema. We also want to be able to look up the specific schema version an event was originally written with.

Wikimedia distributes schema files in git repositories, but we intentionally don’t use git for versioning of those files. We want every version of a schema to be readily available for on demand use by code. Instead of naming files after the schema title, we name them with their semantic version. The JSONSchema title keyword is used to name the full schema lineage, and we keep immutable versioned files in a directory hierarchy that matches the schema title. 

For example, an event schema that is modeling user account creations might be titled ‘coolsoftware/user/create’. The various versions of this schema would live in our schema repository in the coolsoftware/user/create/ directory with semantically versioned filenames like ‘1.0.0.json’, etc. An example schema repository directory tree might look like:

schemas
└── coolsofware
  ├── user
  │  ├── create
  │  │  ├── 1.0.0.json
  │  │  └── 1.0.0 -> 1.0.0.json
  └── product
    └── order
      ├── 1.0.0.json
      ├── 1.0.0 -> 1.0.0.json
      ├── 1.1.0.json
      └── 1.1.0 -> 1.1.0.json

These schemas will be addressed by URIs. To hide the file format extension in those URIs, we create extensionless symlinks to each version, e.g. 1.0.0 -> 1.0.0.json. 

Each schema version will identify itself using the JSONSchema $id keyword set to its specific versioned relative base URI inside the schema repository. The coolsoftware/user/create/1.0.0.json file will have "$id": "/coolsoftware/user/create/1.0.0“. (Using $id in this way also has advantages for reusing schema fragments using $ref pointers.)

Events will have their $schema field set to a relative schema URI that now includes the schema title and the schema version, matching exactly the $id field of the versioned schema. A coolsoftware/user/create event now might look like:

{ 
  "$schema": "/coolsoftware/user/create/1.0.0",
  "datafield1": "value1"
}

Code can use the $schema URI to look up the schema for an event from a local or remotely hosted schema repository.

This versioned schema hierarchy inside of a git repository allows us to decentralize the versioned schemas and refer to them with base location agnostic relative URIs.

jsonschema-tools

Each versioned file in our repository contains a full JSONSchema. But this means that whenever a developer wants to make a new version, they have to copy and paste the full previous version into their new version file and make the changes they want. This could result in subtle copy/paste bugs or undetected schema differences.

Also, every distinct schema lineage file with the same major version must be fully compatible with each other. The only type of change that is allowed for full compatibility is adding optional fields. This means no field deletion and no field renaming (which is essentially a deletion).

Rather than relying on developers to enforce these rules, Wikimedia has developed jsonschema-tools— a library, command line interface and set of tests to manage the schema development lifecycle.

Instead of manually keeping copies of each schema version, jsonschema-tools generates schema version files from a single ‘current’ schema file. A developer can modify the current schema file, update the version in the $id field, and still keep the previous version files around. It will also (by default) attempt to dereference any JSON $ref pointers so that the generated version schema files require no dereferencing or lookups at runtime. jsonschema-tools also exports a set of tests that can be run in your schema repository to enforce compatibility between the versions, as well as rules and conventions. 

Continuing with the coolsoftware/user/create schema example, If a developer wants to add a new field, they would edit the coolsoftware/user/create/current.json file, add the field, and bump the version in the $id field to "$id": "/coolsoftware/user/create/1.1.0". jsonschema-tools will handle generating a static coolsoftware/user/create/1.1.0.json file (as well as some handy symlinks). If your schema repository has been configured properly, running npm test will recurse through your schema repository and ensure that schema versions with the same title are compatible with each other.

jsonschema-tools does a little more than what is described in this post. Check out the documentation for more information.

This all might seem like a lot just to manage schemas, but a big advantage here is that schemas version are decentralized and immutable. jsonschema-tools does move some of the complexity of developing schemas to the developer environment. However, once a schema change is merged, all that is needed for other developers or code to access that schema is to have a URI and a clone of the schema repository (either locally or accessible via http).

Now that we’ve made versioned and compatible schema files accessible, how do we use them to ensure that the events produced into Kafka are valid? Wikimedia has developed an HTTP event intake and validation service and library based on JSONSchema URIs: EventGate. The next post will go into how EventGate works, how Wikimedia uses it, and how you can use it together with your schema repository to build your own custom event intake service.

About this post

This is part 2 of a 3 part series. Read part 1. Part 3 will be published next week.

Featured image credit: Vue aérienne raprochée du grand récif de Gatope et son trou bleu, Kévin Thenaisie, CC BY-SA 4.0

By Pavithra Eswaramoorthy

Every summer, budding technologists from around the world come together to contribute to Free and Open Source software. At the epicenter of this contribution season are Google Summer of Code and Outreachy — remote activities and internship programs with the goal of getting more people involved in FOSS.

Google Summer of Code is a program for university students, and Outreachy is a program that supports members of underrepresented communities in technology. Participants start by completing an application, where they submit a project proposal for a FOSS organization. Selected participants are then paired with experienced mentors for a period for 3 months, and they spend the summer working on their projects.

Experiences and achievements

Wikimedia has participated as a mentoring organization for GSoC since 2006 and for Outreachy since 2013. This round, we received record-breaking student participation with over 130 total proposal submissions from 13 different countries!

In May, Wikimedia welcomed 17 selected participants and kicked-off the community bonding phase with a ‘Welcome Party’ at the Wikimedia Remote Hackathon 2020. 

Participants and mentors worked on projects ranging from test engineering to data science. Webdriver IO, MediaWiki’s test automation framework, was evaluated and upgraded. New features were added to the Commons Android App. Wiki Education Dashboard’s error handling mechanism and efficiency were enhanced. The Page Forms extension, Proofread Page extension, GDrive-to-Commons uploader tool, and WikiContrib tool were improved. 

Some new projects were also developed, including a bot for support on Wikimedia’s Zulip, a tool to correct false depicts claims on Wikimedia Commons, and a tool for automating tasks for Wikimedia databases. 

Learn more about the projects and outcomes:

• Google Summer of Code 2020 project outcomes
• Outreachy Round 20 project outcomes

Participate in the next round

Contributing to Free and Open Source Software can be fun and rewarding. You can work alongside experienced community members, contribute to real-world projects, and witness the impact of your work first-hand. Moreover, there are various other areas to contribute besides code! For instance, every project needs documentarians, designers, and community builders.

Wikimedia always welcomes new contributors. 🙂 To get involved, select an area that is interesting to you: https://www.mediawiki.org/wiki/How_to_contribute. Programs like Google Summer of Code, Google Season of Docs, and Outreachy can provide a platform to accelerate your FOSS journey. Learn about Wikimedia’s outreach programs: https://www.mediawiki.org/wiki/Outreach_programs 

If you are undecided, have any questions, or just want to hang out with Wikimedians, say “Hi!” to us on Zulip!

Note: The initial applications for Outreachy 2020 winter edition are now open. We are also looking for projects and mentors. For more details: https://lists.wikimedia.org/pipermail/wikitech-l/2020-August/093759.html

Thank you, organizers, coordinators, and mentors!

This year brought some unique challenges. The COVID-19 pandemic affected people around the world and there were abrupt changes to personal schedules. Yet, our mentors were always ready to help and support the participants. The program organizers at Google and Outreachy adapted to the changes quickly and kept the programs running smoothly. Their hard work was the primary reason behind the success of this season, so thank you!

About this post

A note about the programs featured in this post: Outreachy is an internship program; whereas Google Summer of Code is considered an activity, see: https://developers.google.com/open-source/gsoc/faq#is_gsoc_considered_an_internship_a_job_or_any_form_of_employment

Featured image credit: Sea otters holding hands, John Robertson, CC BY 2.0

A time of transition

To say that the spring 2020 academic term was unlike any other is a gross understatement. The pandemic abruptly upended normal life, and the students and instructors in our program were no exception. Mid-way through the term, just as most of our students were about to begin drafting their contributions and moving into the article main space, campuses closed nationwide, and students had to return home for the remainder of the academic year. Amid this turmoil, Wiki Education was thankfully able to continue providing uninterrupted support, and we reached out to all of our program participants to see how we could help.

Despite the unprecedented disruptions that the spring 2020 term saw, our students still did outstanding work, and Wiki Education supported 409 courses — our largest number to date. In fact, we quickly learned that the Wikipedia assignment became a lifeline for many of our instructors as they struggled to transition to remote learning. As one instructor wrote, “It actually was a perfect thing that I already had this scheduled and in the syllabus. It gave us an online component before we ‘needed’ it!” Others decided to run a Wikipedia assignment as a way to keep students engaged as their courses went online. We were truly grateful that we could provide some degree of stability for our instructors and students amid a tumultuous time, and despite it all, our students made substantial contributions to Wikipedia.

Across the 409 courses we supported in Spring 2020, almost 7,500 students collectively contributed more than 5 million words to Wikipedia, edited more than 6,000 articles, and created more than 500 entirely new entries. They covered subjects ranging from African American History to Prokaryotic Processes, and their work was collectively viewed almost 270 million times in the term alone.

A source of empowerment

We were also gratified to see that the Wikipedia assignment continued to be a source of motivation and empowerment for our students, as reported by instructors in our post-term survey. One instructor recounted the following anecdote: “Because of the pandemic, our university library was slower than usual. One student, frustrated that one of the books she wanted to read for the Wikipedia assignment was not immediately available at our university library, got on a bicycle and went to borrow a book in a city library in her town. I’ve never seen a student do that. I was happy to hear that, though I tried to hide my reaction in front of the student!” Another instructor reported, “One thing the struck me is I actually had a student came up and thank me for assigning this as their final. I was floored because, I’ve never had a student actually thank me for assigning them work or a final. This student really loved writing for Wikipedia and showing off their work to their family.”

Wiki Education has long been committed to filling in Wikipedia’s content gaps and especially those related to issues of equity, and we’re pleased to see that many of our students were also motivated by a desire to fill in Wikipedia’s equity gaps. “The project,” noted one instructor, “also increased their understanding of representation — who is included, who is not, and whose absence we notice. In many ways this grew out of the understanding that absences need to be addressed and that information is power (the Women in Red project). So thank you.”

Many of our students work on biographies of women to help close Wikipedia’s gender gap. One such student described their experience writing about Andrea Ivory: “Personally, I learned a lot when I was younger just going on random Wikipedia rabbit holes. It was really cool doing a deep-dive into Andrea Ivory’s backstory and working with my teammates to prove that she is, in fact, well known. And, more importantly, her work should be spotlighted regardless of how much media coverage she gets. Keeping my own experiences in mind, I was excited to go live with an article about such an important figure, because you never know who’s Wikipedia rabbit hole session may lead them to know more about Andrea Ivory!”

Just as important as filling in Wikipedia’s equity gaps is making the population that engages with Wikipedia editing more equitable. The students who participate in our program are representative of college campuses more broadly, and roughly 60% of our students are women. More surprising, however, is the number of female instructors who decide to run Wikipedia assignments. About 73% of our new instructors and about 60% of our returning instructors identify as women, a number that is far greater than the roughly 31% of women that make up academia more generally. Wikipedia has long struggled with a fairly homogeneous editing community — composed largely of men — and through our efforts, we are changing the face of who writes Wikipedia and who decides what content should be included on the world’s largest online encyclopedia.

Not simply an assignment, but a way to teach

When considering a Wikipedia assignment for the first time, instructors are often concerned that their students will spend more time learning how to contribute to Wikipedia than the content of the course. To the contrary, learning the ins and outs of Wikipedia can truly complement a course’s broader objectives. According to one instructor, “I started this assignment with the hope of highlighting the difference between academic knowledge (history, in my case) and Wikipedia knowledge. I ended up teaching, more effectively than otherwise, the key processes of history research, instead of teaching students what Wikipedia is. In other words, Wikipedia assignment helped me teach history, more so that a conventional research seminar can do.”

Another instructor remarked, “The Wikipedia project served as a great introduction to Women’s and Gender Studies as it incorporated the material we had discussed in class in a tangible, realizable way. It showed how the theories and practices we covered are not just concepts or academic discourse but are real, applicable, and powerful.”

The Wikipedia assignment transforms students from passive learners to active producers of knowledge. In a time where many feel helpless and powerless to change the world around them, this is no small feat. As one instructor put it, “Wikipedia has become a key pedagogical tool for me in my classes. I still start each semester anxious about how everything will go, but I’m increasingly impressed with what the students accomplish and what they get out of it! It’s a source of community and a source of empowerment and a space of learning.”

We all learned a lot from the challenges of the spring 2020 term, and we know that many of those challenges will persist as we embark on our fall 2020 adventure. What remains constant though is the potential our students and instructors have to truly make a difference through their engagement with Wikipedia.

I added this template {{PositionHolderHistory|id=Q**}} to all the items for an office in my African politicians project.  You find the template on the talk page and like the Listeria lists, they show past and present office holders.

I still prefer my method of including the "red links" in Wikidata but it is a wiki and there is so much more to do.. What I started to do with office holders from Togo is that for those I will not link to predecessors and successors, I will at least show the dates they were in office. 

It looks much better in Listeria too. 

Thanks, GerardM

01/09/2020-07/09/2020

About us

• Our editorial system now gives you the opportunity to post your articles directly into our system, and to suggest an initial formulation. We may make slight editorial changes to adapt the text to our language style. Translation into 12 languages will be done by us.

Mapping

• The French ‘Project of the Month’ > , has a graphically prepared interim balance of what has been achieved. It still has 18 days to run and aims for the complete mapping of defibrillators (AED). PanierAvide gives > , in addition, a few tips for further work.
• In September 2010, the Berlin-based association Sozialhelden e.V. launched > the Wheelmap.org project. Ten years later, Wheelmap has become the largest online map of wheelchair accessible places with over one million entries. In September, the team invites you to join in the celebrations under the motto ‘Wheelmap turns 10’. The campaign is taking place in the Wheelmap social networks and as a digital mapping action. They are also launching a small mapping campaign with the aim of getting people to each evaluate ten places on the Wheelmap that still have unknown wheelchair accessibility.
• Andrew Harvey proposes the new tag shelter_type=rock_shelter to mark the difference between the already existing natural=cave_entrance and a shallow cave-like opening at the base of a bluff or cliff, which may be used to protect yourself from the weather.
• In his blog post > , Robhubi deals with different types and names of watercourses that lead to mills in Austria and Germany. He evaluates entries in OpenStreetmap. WSHerx rates the publication as ‘impressively thorough work and convincing argumentation’.

Community

• On 1 September, Amazon Web Services (AWS) released an episode of their documentary series Now Go Build, which highlighted the work done by the Humanitarian OpenStreetMap Team in the Philippines, especially in mapping the town of Guagua, in the province of Pampanga.Several members of the OSM-PH community, however, observed that there were missing and problematic narratives in the video related to the story it tells of geospatial and humanitarian workers in the country. Therefore, they prepared and released this statement (pdf).
• Martin Koppenhoefer asked on osmf-talk whether Brexit will have an effect on OSM and wondered if the foundation could be moved to another European country. The core database, website and API are already hosted in the Netherlands (since late 2018), but there is still some user data stored in the United Kingdom (Forum, Piwik, Foundation wiki, mailing lists). Additionally AWS is used for some data storage needs; the data is primarily stored on AWS Ireland.
• AI is not a magic bullet. Even for a company as resourceful as Facebook the results of an image recognition algorithm still needs to be manually verified. Some mappers using the results of MapwithAI, bypassing the upload limits, mass uploaded without reviewing the results. As a result users have discovered strange ‘roads’ on Rudy Map, the most important offline map dataset in Taiwan, and have reported them in the Facebook user group. Cartographers of a mountaineering group have also had to remove non-existent roads in the mountains, or correct streams without water in the dry season, which were wrongly classified as roads.
• Prajwal Shrestha, a student of the Institute of Agriculture and Animal Science-Lamjung, Tribhuvan University (Lamjung Campus), Nepal, shared his excitement about joining the OSM community through the creation of a new YouthMappers chapter (Agri-Mappers Lamjung) on his university campus.
• Facebook has released a third update to Daylight, their complete, downloadable preview of OpenStreetMap data (57 GB). Michal Migurski gives some insights in his blog and provides relevant links.

OpenStreetMap Foundation

• Dorothea Kazazi gave us a brief overview about the recent OSMF reports: Volunteer WGs, the Board of Directors and the Committees, Local Chapter applications and related news, Membership statistics and added a link to previous reports.

Events

• The 2020 Pista ng Mapa (Festival of Maps) organisers from the OpenStreetMap and FOSS4G communities in the Philippines are calling for presenters and workshop proposals on Open Data, Mapping, and FOSS4G for presentation at their 2020 conference this November. The call is open until 30 September. Also, this month, they are holding a map and poster making contest with the theme ‘Mapa para sa lahat’ (Maps for everyone). Check out their website for details. Prizes await winners in two categories: students and professionals.
• Alessandro Sarretta presented a poster entitled ‘OpenStreetMap: an opportunity for Citizen Science’ at the ECSA 2020 conference, showing the many ways OpenStreetMap and citizen science can help each other.

Humanitarian OSM

• Jean-Marc Liotier criticised HOT’s use of the number of mapped buildings as a measure of the ‘state of OpenStreetMap in Africa’. At the same time, he acknowledged that criticising bad metrics is easy, but finding good ones is difficult and he agreed to participate in an ‘open working group on OpenStreetMap metrics’.
• Taichi Furuhashi (User MAPconcierge) looked back on the crisis mapping conducted in response to heavy rain in Japan and summarised points to be improved. Over 1800 people participated in the task, and over 65,000 buildings were mapped.

Maps

• Technion (Israel Institute of Technology) has developed a mapping system to assist blind people in navigating cities.

Open Data

• The Croatian State Geodetic Administration has published > new aerial imagery for the eastern and southern parts of Croatia taken in 2019, and extended new topographic 1:25,000 map coverage.

Software

• Ulf Buermeyer has created MapAlarm, an iOS app that allows you to save POIs. According to his own statements MapAlarm should become a ‘personal placebook’. The application is in beta testing and Ulf is asking for help.

Programming

• At the Google Summer of Code, Kishan Tripathi, K Rahul Reddy, Rebecca Schmidt and Daniel Weaver worked on OpenStreetMap.
  • K Rahul Reddy documented his project about search suggestions on osm.org in detail on his blog.
  • Here you can find the code for indoor tagging by Rebecca Schmidt, Kishan Tripathi and Daniel Weaver, whose blog post we reported on last week.

Releases

• The changelog for the stable version #17013 of JOSM lists, among many others, the addition of the Serbian language with Latin script, GPX routes as a separate layer, and a dark mode via the plugin FlatLaf as major changes.

Did you know …

• … OSM Streak? Do you still need a few days to get your free membership at the OSM Foundation? Ilya Zverev’s Telegram bot OSM Streak reminds you with a small daily task.
• … there is a list of the OSM mailing lists? However, many of them are orphaned.
• … OSM Ireland has its own site for mapping tasks that can be solved from home? The focus is on buildings.

Other “geo” things

• Allan Mustard says the article by Maite Vermeulen, Leon de Korte and Henk van Houtum in The Correspondent presents an ‘excellent case study of political bias in maps’.
• When US companies were ordered to stop working with Huawei, Google Maps was one of the apps that was dearly missed. Now the Chinese smartphone company has released TomTom’s navigation app on Huawei AppGallery. With OsmAnd, Here WeGo and maps.me there were already some alternatives available in the store.

Upcoming Events

Where	What	When	Country
Potsdam	147. Berlin-Brandenburg Stammtisch	2020-09-10
Munich	Münchner Treffen	2020-09-10
Zurich	121. OSM Meetup Zurich	2020-09-11
Leoben	Stammtisch Obersteiermark	2020-09-12
San José	National Day of Civic Hacking	2020-09-12
Ashurst	Trek View New Forest Pano Party	2020-09-13
Montrouge	Rencontre des contributeurs de Montrouge et alentours	2020-09-14
Cologne Bonn Airport	131. Bonner OSM-Stammtisch (Online)	2020-09-15
Lüneburg	Lüneburger Mappertreffen	2020-09-15
Salt Lake City / Virtual	OpenStreetMap Utah Map Night	2020-09-15
Kabul / Online	Why OSM and how to Contribute into it on Software Freedom Day 2020	2020-09-18
Nottingham	Nottingham pub meetup	2020-09-22
Bratislava	Missing Maps Mapathon Bratislava #9	2020-09-24
Düsseldorf	Düsseldorfer OSM-Stammtisch	2020-09-25
Salt Lake City / Virtual	OpenStreetMap Utah Map Night	2020-09-29
Alice	2020 Pista ng Mapa	2020-11-13-2020-11-27
Alice	FOSS4G SotM Oceania 2020	2020-11-20

Note: If you like to see your event here, please put it into the calendar. Only data which is there, will appear in weeklyOSM. Please check your event in our public calendar preview and correct it, where appropriate.

This weeklyOSM was produced by AnisKoutsi, GOwin, MatthiasMatthias, Nordpfeil, Polyglot, Rogehm, TheSwavu, derFred, richter_fn.

A blog post about the Wikimedia Design style guide was recently published.

The origin was a conversation with David Golberg about the construction of open participatory design systems and some parallelisms with architecture.

I just published a VS Code language extension to support syntax highlighting for Stuttgart Finite State Transducer (SFST) formalism to VS Code. Download and install the extension Source code I learned how to write a language extension when I attempted the opentype feature file support. So I thought of applying that learning to SFST which I regularly use for the Malayalam morphology analyser project.

Are you looking for a mobile and responsive skin for your wiki? I got you covered with a comparison of the best skins.

I believe that your wiki should naturally serve mobile and responsive devices from the start, but it can be difficult to find a skin to fit this purpose. Do these mobile friendly skins even exist? Let’s find out and take a look at our best picks for MediaWiki you should consider.

Overview

Even today, there are not many choices when it comes to a mobile and responsive skin. I’ve done the research for you and in this blog post, I will look at the available alternatives serving the content of your wiki best for mobile devices.

All of the mobile and responsive skins to MediaWiki I will mention in this blog post both integrate with Semantic MediaWiki and VisualEditor. According to my opinion these two requirements need to be met. In addition, these skins are under active development, meaning that support for new major releases of MediaWiki is added within a short period of time. The latter is good to know if you would like the next major upgrade of your MediaWiki instance to be smooth when it comes to the skin used.

Let us now look at the selection of mobile and responsive skins for MediaWiki I made for you.

Chameleon

The Chameleon comes by default with a layout offering two navigation bars enclosing the wiki’s content at the top and bottom, with the first holding the menu for actions on the right and the latter holding the tool menu. This is different to what you are accustomed to by the classical wiki skins for MediaWiki but I do not think this is bad.

It is the most versatile and flexible skin when it comes to customizing the look and feel of your wiki. Moreover it can easily be extened with further features from the Bootstrap framework. If you are into creating an unique impression of your wiki then this skin is your choice.

Foreground

The Foreground skin serves a fixed layout providing one navigation bar at the top above the wiki’s content which also holds the tool menu. The menu for actions is located at the top right of the content area.

It is one of the skins with a clear and straigt appearance and serves by default the features provided by the Foundation framwork. If you are happy with the layout and just want to do a couple of changes to the CSS then you should pick this skin.

Pivot

The Pivot skin serves a fixed layout resembling that of the well-known classic Vector and Monobook skins with a sidebar on the left of the screen. The user menu is toggled in and out on the right side of the screen. The menu for actions is located at the top right of the content area.

This is a mobile and responsive skin offering the classic wiki appearance and serves by default the features provided by the Foundation framwork. If you are into the classic wiki layout and just want to do a couple of changes to the CSS then this skin is your pick.

Tweeki

The Tweeki skin serves a flexible layout by default providing one navigation bar at the top above the wiki’s content which also holds the menu for tools as well as for actions. The latter with the exception of the action for editing which is placed as a button on the top right of the content area.

It is a skin with a clean aesthetic layout by default bringing the features provided by the Bootstrap framwork. If you like the layout provided by the Tweeki skin and you just want to do a couple of changes to the CSS, then this skin is definitely something for you. If you would like to build intensely upon the skin, it is also a very nice choice.

Adoption

Let us now look at the basic data like usage numbers, web framework used, and other information:

Skin	Usage	Since	Framework	Demo site
Chameleon	215	2013	Bootstrap 4	no
Foreground	208	2014	Foundation 5	yes
Tweeki	68	2016	Bootstrap 3	yes
Pivot	38	2017	Foundation 5	yes

The data was sourced from WikiApiary.com, MediaWiki.org and GitHub.

The usage numbers of these mobile and responsive MediaWiki skins do not convey a relevant volume of serious adoption. However, we need to remember that private wiki instances running MediaWiki are not covered at all. From experience, these are much more likely using one of the skins mentioned here. Thus, these numbers do, indeed, provide a pretty good overview of how usages compare by skin. Moreover, their overall adoption is steadily increasing since their inception.

Interestingly, but not surprisingly, none of the mobile and responsive skins mentioned here is deployed on wikis run by the Wikimedia Foundation, the organization behind Wikipedia. One reason for that is due to the skins’ focus on being mobile and responsive, they provide features that may break content on Wikipedia. Moreover these features are even unwanted within the scope of Wikipedia. Think of features like accordions, carousels or tabs here. Another reason is that they were not developed by the Wikimedia Foundation, thus not assuring direct control over their code base and release process, which is also an important factor for Wikipedia and related projects.

This leads to the questions: Which mobile skin is used for Wikipedia, and why wasn’t it considered in this blog post? Well, the skin is called Minerva Neue, which is best used with the MobileFrontend extension. They piggy-back on the classic MediaWiki skins we all know - Vector and MonoBook. In my experience, this solution to a mobile skin for MediaWiki is generally viewed by users with a resigned acceptance. Also it lacks the possibility for adaptation to individual wiki’s requirements.

Chameleon is one of my top mobile and responsive skins for MediaWiki.

The Chameleon skin is currently, just by a bit, the most popular mobile and responsive skin for wikis using MediaWiki. It comes by default with a layout offering two navigation bars enclosing the wiki’s content at the top and bottom, with the first holding the actions menu on the right and the latter holding the tool menu.

It is the most versatile skin around, thus truly honoring its name. Additionally, it is the only skin that really reconciles a website with the wiki way. Typically, one can easily see that a website is a wiki using MediaWiki. However, if you look at some implementations utilizing Chameleon, you would not at all guess that it is a wiki run with MediaWiki under the hood. MediaWiki has undoubtedly gone a long way to get here.

Stephan Gambke developed the Chameleon skin in 2013, and it was first released in October that year. It uses the Bootstrap framework developed by Twitter, and it has utilized the most recent version of 4 of Bootstrap since May of 2019.Currently, the Chameleon skin is being maintained by Professional.Wiki.

The Chameleon skin can easily be extended with support for FontAwesome (v5.11.2) and Bootstrap components such as accordions, modals or popups and many more. Another great way of adapting the Chameleon skin is the option to make use of predefined Bootstrap themes created by members of the Bootstrap community. You will still need to tweak these to integrate better with MediaWiki, but most of the basic look and feel is there. Also, the XML structure defining the layout of the wiki with the placement of menus, sidebar and logo, etc. can easily be adapted allowing for much more general flexibility than all other skins around which come with a fixed layout.

The skin’s documentation is accessible via it's page on MediaWiki.org.

If you are into creating a unique impression of our wiki and do not fear doing the required adaptations and changes to the CSS and XML, then there is no way around this skin. I may be biased, but I can live very well with that.

Example sites using the Chameleon skin

• Secret Wiki
• text-menü
• Semantic MediaWiki

Check out my other favourite responsive and mobile MediaWiki skins.

Foreground is one of my top mobile and responsive skins for MediaWiki.

The Foreground skin to MediaWiki is currently the second most popular mobile and responsive skin for MediaWiki. It comes with a layout providing you with a straight and clear view of the content arranged below a top menu. The menu for actions is located on the top right of the content area, whereas the tools menu is integrated into the top navigation bar.

Jamie Thingelstad and Garrick van Buren developed the Foreground skin in 2014 and it was first released in July that year. It uses the Foundation framework developed by Zurb. Since May 2018, it has been utilizing version 5, whereas the most current version 6 of the framework is not supported. Currently it is being maintained by Tom Hutchison.

The Foreground skin comes by default with support of Font Awesome (v4.7.0), several components like accordions, grid views tabs or an image slider, and it allows for a couple of changes via configuration. However, the general layout of the skin is set. It natively provides integration with share and follow buttons serviced by AddThis. When it comes to the general look and feel, you will most probably want to change the default color scheme as well as tweak the proportions of the font elements, which are overall a bit unbalanced.

A demo wiki together with documentation is available at foreground.wikiproject.net.

If you are happy with the layout provided by the Foreground skin and you just want to do a couple of changes to the CSS, then this skin is something for you.

Example sites using the Foreground skin

• The Grindhouse Cinema Database
• Wikimanche
• Le Dico des Ados

Check out my other favourite mobile and responsive MediaWiki skins.

Foreground is one of my top mobile and responsive skins for MediaWiki.

The Pivot skin to MediaWiki is a less popular mobile and responsive skin for MediaWiki. Still, it should be considered as a viable option to choose from. It comes with a layout resembling that of the well-known classic Vector and Monobook skins with the sidebar on the left of the screen. The user menu is toggled in and out on the right side of the screen, which makes it unique in the MediaWiki world. The menu for actions is located on the top right of the content area.

Tom Hutchison developed the Pivot skin in 2016, and it was first released in March 2017. He has continued to maintain it since. Like the Foreground skin, it also uses the Foundation framework developed by Zurb. Similarly, it has utilized version 5 of the framework since February 2018. The most current version 6 of the framework is not supported.

By default, the Pivot skin comes with several components like accordions, grid views, tabs or an image slider, and it supports Font Awesome (v4.7.0). The general layout of the skin is set, but a couple of changes via configuration are possible including the integration with share and follow buttons serviced by AddThis. When it comes to the general look and feel, you will most probably want to change the default colour scheme, which otherwise is in need of getting used to.

A demo wiki together with documentation is available at pivot.wikiproject.net.

If you are looking for a mobile and responsive layout that resembles the layout of the Vector or the MonoBook skin and you just want to do a couple of changes to the CSS, then this skin is your pick.

Example sites using the Pivot skin

• International Society of Genetic Genealogy Wiki
• Louisville Punk/Hardcore History
• HüPedia

Check out my other favourite mobile and responsive MediaWiki skins.

Tweeki is one of my top mobile and responsive skins for MediaWiki.

The Tweeki skin to MediaWiki is also a less popular mobile and responsive skin for MediaWiki, but it is a viable option when considering a skin. It comes with a straight and clean aesthetic layout with the content arranged below the top navigation bar. The menu for actions, except for the edit action, is located on the right within the top navigation bar and was merged with the toolbox. The edit button was placed next to the content area.

Tobias Haider developed the Tweeki skin from February 2013 as a fork of the then Strapping skin created by Garrett LeSage in September 2012, and he maintains it since. The first official release of the Tweeki skin was, however, only made in October 2016. Similar to the Chameleon skin, it also uses the Bootstrap framework developed by Twitter. However, it utilizes version 3 of the framework, which is not the latest one.

By default, the Tweeki comes with several components like accordions, modals, tabs or tooltips. The general layout of the skin is set, but it can be adapted to quite some extent via configuration including custom styles. When it comes to the general look and feel, the default is already really nice to start your wiki with.

A demo wiki together with documentation is available at tweeki.kollabor.at.

If you like the layout provided by the Tweeki skin and you just want to do a couple of changes to the CSS, then this skin is definitely something for you. However, if you would like to intensely change the look and feel of the wiki Tweeki is also a basis to build upon with your individual adaptations too.

Example sites using the Tweeki skin

• A Wiki of Fire and Ice
• Tango Wiki
• Le violon de Jos

Check out my other favourite mobile and responsive MediaWiki skins.

By Andrew Otto, Staff Site Reliability Engineer

In the past few years, event-driven architectures (EDAs) have been getting a lot of attention. When done right, they help fulfill some of the promises of service-oriented architectures (SOAs) while mitigating many of their headaches. Event sourcing, coupled with complex event processing in particular, allows data to be materialized into services in the format they need when they need it. This recent data architecture trend can mostly be attributed to the success of Apache Kafka, which enables these kinds of architectures at scale. Confluent, founded by the creators of Kafka, have done an amazing job writing about and evangelizing EDAs on their blog, so we won’t try to expound on them any further. Instead, this 3-part series will focus on how Wikimedia has adapted these ideas for our own unique technical environment.

The Wikimedia Foundation has been working with event data since 2012. Over time, our event collection systems have transitioned from being used only to collect analytics data to being used to build important user-facing features. A few years ago, we began an effort to refactor our special-purpose analytics events system to one that closely resembles what Confluent calls a ‘stream data platform.’ However, even though most of Confluent’s components are free to use, we chose to implement some of our own. This article will first explain this decision and also briefly describe the Open Source components we developed to build Wikimedia’s Event Data Platform.

Confluent Community License

When Wikimedia began designing our event platform, all of Confluent’s free components were Open Sourced under the Apache 2.0 license. At the end of 2018, Confluent switched all its free software to a new custom Confluent Community License, which is not strictly a Free Open Source software license. One of the Wikimedia Foundation’s guiding principles is to use Free Open Source software in the software and systems we develop. Our decisions to avoid Confluent components were not originally influenced by the CCL—we began our system design process before they changed licenses—but in hindsight they proved to be good choices. (Aside: we did want to use Confluent’s Kafka Connect HDFS and now can’t due to the CCL, but I’ll save that story for another day.)

Event schemas

The main differences between Wikimedia’s and Confluent’s event stream platforms are all related to event schemas. In an event-driven architecture, events must conform to a schema in order for the various disparate producers and consumers to be able to communicate reliably. Schemas define a contract for the data used by many services, just like an API specification would in typical SOAs. They are a required component of building dependable and extensible EDAs.

What about Avro?

Confluent’s stream data platform uses Apache Avro for event schemas and for in-flight data serialization. Avro is a natural choice for organizations that are already heavily JVM-based. Avro is language-agnostic, but the software ecosystem around it is oriented towards the Java virtual machine. In addition to needing an Avro implementation in a language to use Avro data, an Avro library also needs an Avro schema in order to serialize and deserialize Avro binary data records. Outside of a streaming context, Avro data is usually serialized in binary files. These files contain a schema header which instructs Avro readers how to read the data records in the rest of the file.

However, in a streaming context, where every message in Kafka is a single data record, how can a consumer get the Avro schema it needs to deserialize the binary record? Confluent solves this problem with its Schema Registry. The Schema Registry is a centralized service that stores Avro (and as of April 2020, JSONSchema and Protobuf) schemas. To produce an Avro message, you must use a custom Confluent serializer that wraps the binary Avro message with an envelope containing an opaque numeric schema id registered with your running installation of Schema Registry. When consuming the message, you must also use a custom Confluent deserializer that knows how to unwrap the message payload to read the schema id, and then look up that schema id from the Schema Registry.

That’s a lot of service coupling just to produce or consume data. Any time you want to produce or consume any data from Kafka, you must have on hand a Confluent client implementation that is configured properly to talk to the running Schema Registry service. This coupling might be acceptable in a closed JVM shop, but it is untenable for the Wikimedia Foundation.

Making choices

Wikimedia’s software development model is decentralized. Volunteer developers must be able to contribute to our software just as Wikimedia staff does. We try to build forkable software, data, and systems. For example, our production helm charts and puppet manifests are Open Source. If someone tried hard enough, they could reproduce the entirety of the Wikimedia technical infrastructure. Using Avro and the Confluent Schema Registry would make this difficult. Yes, we could find ways of transforming data and formats for public consumption, but why make our lives harder if we don’t have to?

JSON is arguably more ubiquitous than Avro. But it is only loosely schemaed. The name of every field is stored in each record, but types of those fields are not. JSONSchema is commonly used to validate that JSON records conform to a schema, but it can also be used to solve data integration and conversion problems (AKA ETL) as long as the schema maps well to a strongly typed data model.

In 2018, we went through a Wikimedia Technical RFC process to decide if we should commit to the Avro route or stick with JSON, and ended up choosing JSON mostly due to the fact that most consumers of event data would not need any special tooling to read the data. However, by choosing JSONSchema instead of Avro, we walked away from Confluent’s Schema Registry, as at the time it only supported Avro schemas. We also missed out on a really important feature of Avro: schema evolution. 

We were left needing to implement for JSON and JSONSchema two features that are built into Confluent’s default stream data platform components: Schema evolution and schema distribution. We noticed that we weren’t the only ones that needed tools for using JSONSchemas in EDAs, so we decided to solve this problem in a decentralized and open sourced way. Schemas are programming language-agnostic data types. We felt that they should be treated just like we treat code, so we chose to distribute schemas using git. We also chose to enforce compatible schema evolution just like we’d check code changes: using code tests. The outcome was jsonschema-tools, a CLI and library to manage a git repository of semantically versioned JSONSchemas.

We also needed an easy way for developers to produce valid event data into Kafka. Confluent has a Kafka REST Proxy which integrates with their Schema Registry to do this. We wanted a generic event validating and producing service that would work with decentralized JSONSchemas. EventGate was born to do just that.

Next up

The next post in this series will explain how we manage JSONSchemas for streaming event data using jsonschema-tools. After that, expect a post on how EventGate interacts with a schema repository to validate and produce events, as well as Wikimedia’s specific EventGate customizations and features.

About this post

This post is the 1st of a 3 part series. Read the 2nd post here.

Featured image credit: Dobson Stream by Wharfedale hut with the moon, Mt Oxford area, Canterbury, New Zealand, Michal Klajban, CC BY-SA 4.0

By Sara Thomas, Scotland Programme Coordinator.

At the beginning of the lockdown period in the UK, the Programmes Team, along with our network of Wikimedians in Residence, got to work on exploring how we could continue to support our community and partner organisations. We discussed ways in which we could offer engagement with the Wikimedia projects as lockdown activities (the National Library of Scotland’s work on WikiSource is an excellent example of how this worked out) for both staff and in terms of public engagement, as well as in the fight against mis- and disinformation surrounding COVID-19.

Back in November we had engaged a new trainer, Bhav Patel, to deliver our Train the Trainer course in Glasgow, Scotland. In light of the success of that programme, we decided to approach him again.

We held two brainstorming sessions, inviting all of our existing trainers to attend. Turnout for these was encouraging, and we were happy to see some familiar faces as well as some individuals whom we had not seen for a while. These sessions were intended to capture both appetite for training, and training needs. We anticipated that as we were approaching existing trainers that there may be less demand for content which pertained to design, however this was not wholly the case. We were also conscious that the high standard of Bhav’s training in the past concerning training design might be beneficial for our existing volunteers. There was significant interest in how to convert existing training methods to the online format, the “tips & tricks” of using specific tools, and liaison with partners in advance to properly assess training needs. There was also some trepidation around how to give individual support over video conferencing, and adapting to feedback in-session in the context of having no physical feedback from the room.

Following these sessions, and further discussions with Bhav, we held two sets of three sessions, delivered at varying times of day to accommodate the existing commitments of volunteers. The first of these sessions, led by Bhav, was titled “Going Online”, and focussed on the move from onsite to online design and delivery of events, taking a platform-agnostic approach. The second was “Tools, Tech & Event Management”, focussing on tips, tricks & a variety of online conferencing and supporting tools, as well as overall event management, and was led by myself.  The third session was an opportunity for practise and feedback, offered to all participants as a way to test out new tools and ideas they had for leading online training. In total, 14 trainers attended across the six sessions, with positive feedback. The two training sessions were recorded for future use, and a pool of accompanying resources created. These were sent to all participants, to the wider volunteer trainer pool, and are being kept as a resource for partner organisations, Wikimedians in Residence, and future volunteers.

Reflecting on this process, we noted a few things. Firstly, that just because we deal with an online resource does not mean that we are automatically prepared to deliver online. Secondly, that the online space opens up opportunities as well as presenting barriers – trainers’ geographic location matters less, for example. Thirdly, that we were able to engage volunteers with a variety of time commitments, from a wider demographic, and that the pressures on their time likely reflect those of our various audiences. Lastly, that ongoing investment in existing volunteers helps to reignite engagement.

Once again, a huge thanks to our volunteer trainers for the time and energy that you give to the movement; it is massively appreciated!

When you want to find a picture for him and you know his name in Russian, you can use Special:MediaSearch. You can also find him with اماما_مبابازى. 

One of the positions Mr Mbabazi held is Justice Minister of Uganda. English Wikipedia has a category for these Ministers, at this time two people are included but not Mr Mbabazi. There are ten Ministers missing in the category. Mind you, it is English Wikipedia that has a list that made my work at Wikidata possible!

On the talkpage for the Wikidata item for Justice Minister of Uganda, I added the {{PositionHolderHistory}} template. A bot updates the information every day and it adds comments on the quality of the information. This makes it easy to add positions of interest to a watchlist. 

On my Africa project you find Listeria lists for African political positions. It is duplicated to several Wikipedias and once a Wikipedia is synchronised it will show information like the lists that include Mr Mbabazi. 

One day all the data will be complete and up to date. In the meantime it is a "work in progress" and you are kindly invited to check the information out, find its shortcomings and make updates where necessary.

Thanks,

       GerardM

I just published a VS Code language extension to support OpenType feature files in the Adobe “AFDKO” format. The extension provides syntax highlighting and code snippet support. (Screenshot From Amiri font) The syntax highlighting patterns for AFDKO is based on the opentype-feature-bundle for Atom Editor by Kennet Ormandy which is based upon Brook Elgie’s original Textmate bundle. The code snippets are based on the snippets prepared by Simon Cozens for AFDKO-SublimeText

Adriana Bankston is a Principal Legislative Analyst at University of California. She recently took the 500 Women Scientists Wiki Scholars course and reflects on her experience in this guest blog post. This post represents the writer’s personal views and not the views of their employer, University of California.

I’m a former bench scientist who transitioned into science policy, and consequently operates at the intersection between academia and policy. As an academic, I was really interested in the environments in which researchers work. To this end, I advocated for women in science by various means, including by organizing professional development activities with a local affiliate group of Association for Women in Science (AWIS), and participating in local outreach activities locally through 500 Women Scientists and others. In doing these activities, I realized how important it is for girls to have role models in science, and looked for ways in which I could show girls of different ages what a scientist looks like. 

My own PhD advisor was also one of two female faculty in the department. She has since moved on from academia, but she inspired me to be bold and stand out despite the circumstances. She also taught me that it’s really important to give back to the community, and I think that drove a lot of my advocacy for women in science. In my transition from science to policy, I maintained an interest in diversifying the research enterprise, particularly after experiencing first hand how the gender gap in science is greater at higher levels on the career ladder. 

During COVID-19, I sought ways to enrich my own professional development, and happened to see a call for applications for the 500 Women Scientists Wiki Scholars course on social media. I had never edited Wikipedia before, but I was eager to make an impact for women in science. I was really excited when I was accepted into the course. My motivation for enrolling was to learn a new skill, but also through editing pages of women in science, I could make a difference in increasing the visibility of women in science. 

The first Wikipedia article I worked on was Sandra Schmid, whom I had previously met in person and long admired from my days in the laboratory as someone who had made a significant impact in her field. She was also an advocate for improving the research system by changing mentoring and hiring practices, as well as by changing the system through improving postdoc training. In short, she was another role model for me both scientifically and professionally in research, and someone whom I felt more people should know about through Wikipedia.

Sandy’s article was an easy one to start with given I was already familiar with the topic, and I considered it notable. During the course, however, I also challenged myself to learn Wikipedia by picking another kind of article which required a lot of improvements initially (Brooke Borel), so that I would learn multiple editing skills by correcting different types of information that was either wrong or missing. 

Notably, while taking the course, I was able to engage in the #editWikipedia4BlackLives #HackForBlackLives event on June 10, in observance of the #Strike4BlackLives and #ShutDownAcademia #ShutDownSTEM as part of a group of 200 people. Altogether, this group created over 50 articles, edited over 480, added over 100,000 words—and in just a few days, those articles had been collectively viewed over 500,000 times. 

By this point, I had been a few weeks into the course, and felt more confident in my edits. Inspired by this event, I was excited to try my hand at editing articles that I felt could have an impact in the community by elevating voices of People of Color in science. I chose to edit articles on Shirley Malcom, Melina Abdullah and Lori White, which taught me a lot. It was a really powerful experience to participate in this collective day of action, and I was glad to utilize Wikipedia editing skills for this purpose. 

As a scientist, I know that we always look for reputable sources, and in the past I might not have considered Wikipedia as such, partly because academics are mostly trained to use peer-reviewed journal articles of high impact as sources to cite for our work. This course gave me a new appreciation for the power of Wikipedia, ways in which it can impact different cross-cutting communities and identities around the world, as well as how we can leverage its accessibility to increase diversity, equity and inclusion in science and beyond.

In closing, I’m glad I took the course and I appreciate the chance to make a difference for women in STEM through the WikiProject Women Scientists, in particular as it relates to diversifying the research pipeline. To this end, I wanted to point out two pages, one is the WikiProject Women in Red looking to create pages for women in science that don’t yet exist, and the second is the List of African American Women in STEM Fields, both of which are useful tools to utilize for making a powerful impact in science. 

Since the course ended, I have sought to utilize these skills in my current field by editing relevant pages such as a science policy page, in order to educate the community on important issues in this area and how they can contribute. 

Interested in taking a course like the one Adriana took? Visit learn.wikiedu.org to see current course offerings. Another 500 Women Scientists Wiki Scientists course is now enrolling!

25/08/2020-31/08/2020

Mapping

• [1] Since 4 August the French community has been discussing > the project of the month for September (defibrillators) in detail on the mailing list talk-fr. Anyone wishing to participate can now read up on the wiki > what to do and which tools are available. The project includes > daily evaluations.
• The Open Maps team from Microsoft announced they will start road editing in Colombia. They will use available street-level and aerial imagery to add where possible but there will be no automatic imports, algorithms or robot edits to improve the map.
• RobJN published two proposals: ref:GB:usrn=* to refer to the unique identifiers for every street and ref:GB:uprn=* to refer to the unique identifiers for every addressable location in the United Kingdom. The voting for both is open until 13 September 2020.
• OpenStreetMap hit 90 million changesets. The changeset was done using RapiD by Diana IRM-ED, an Indonesian mapper paid by HOT.
• Christopher Beddow is concerned about the ‘OpenStreetMap undated imagery crisis’ and he documents it with two pictures on Twitter.

Community

• In early October, the Wikimedia Foundation’s OSM-based public tile service will be shut down. The reason for this is abuse; the majority of traffic has nothing to do with the Wikimedia projects or comes from commercial websites, writes Erica Litrenta on the Maps-l mailing list of the Wikimedia Foundation.
• OpenStreetMap US announced that Connect 2020 will be held online from 29 to 31 October. The call for participants is now open and everyone is invited to apply with proposals related to OpenStreetMap.
• The Unite Maps Initiative seeks a new logo and has announced a competition for one. The deadline for entries is 16 September 2020.
• In the run-up to the 2020 State of the Map conference, OSM Africa surveyed OpenStreetMap community leaders across the continent. Geoffrey Kateregga looks at the results and offers a deep dive into the state and trajectory of OpenStreetMap communities in Africa.
• Changeset 90148752 briefly turned Great Britain into a giant beach and was even visible on the map tiles for about an hour before it was reverted.
• As OSM users of printed maps and OsmAnd Live, the fire fighters in Cordoba, Argentina, requested assistance in mapping roads, waterways and other elements. The community launched a mapathon and in three days they added a lot of information in a very wild area. The firefighters confirmed > that OSM’s detailed maps help save forests.

OpenStreetMap Foundation

• The OSM Foundation is funding the development of Nominatim, osm2pgsql, and Potlatch 2.
• The minutes of the OSMF board meeting in August are online.

Events

• Members of the Italian OSM community in the Piedmont region are organising > an event in the mountains on 19 and 20 September. The weekend includes presentations, mapping and sharing OSM experiences. To participate you must fill in this form > .
• The next Geomob, organised as always by Ed Freyfogle and Steven Feldman, takes place on 16 September at 6.00 pm UTC (20:00 CEST) online. Speakers will be Jenny Allen, Maps Team Lead at Elastic; Ian Hannigan, founder of Formation; Olivier Cottray, discussing using GIS for demining and Hanc Naum, founder of PinApp.
• State of the Map Japan 2020 > will be held online on Saturday 7 November. The organisers are calling for volunteers and presentations.

Education

• All 35(!) worksheets of OpenSchoolMaps > are now available not only as downloadable PDFs but also as web pages. The materials are available in English as well.
• A paper released by the Department of Urban Planning and Spatial Analysis, University of Southern California, uses OSMnx to analyse how urban evolution, planning, design, and millions of individual human decisions shape cities.

Maps

• Frank Schmirler now also offers his ‘Light of the Seas’ in the Portuguese and Russian languages. It is really worth taking a look at this impressive map to explore new shores.

Software

• Pieter Vander Vennet announced MapComplete on talk, where you can now create your own theme with the ‘new easy-to-use editor’. If you have over 500 changesets, you can create a theme from scratch which can be shared via URL. Please visit GitHub to see some results such as cyclofix or the Open Bookcase Map.
• Over the past three months DWeaver worked on OSM2World for the Google Summer of Code. The aim of his project was to add support for indoor tagging, allowing the rendering of indoor data in 3D. This includes basic features such as rooms and corridors, windows and doors as well as objects that may span levels such as stairs and barriers. The code that he produced can be seen in these three pull requests: 1, 2, 3.

Releases

• The GIScience Research Group of HeiGIT, the Heidelberg Institute for Geoinformation Technology, announced that it is now possible to calculate long distance routes using all bike and pedestrian profiles, whilst taking into account restrictions such as avoiding ferries, with the newest update of OpenRouteService.

Did you know …

• Martijn van Exel published his stream on how to use the Meet Your Mappers tool.

OSM in the media

• Buzzfeed contributors and reporters have investigated the regions that were blanked out on Baidu’s maps by comparing with other data sources including OpenStreetMap. They have used those locations to find a network of buildings in Xinjiang bearing the hallmarks of prisons and internment camps. They explained their methodology in this article.

Other “geo” things

• China released > a new version of its standard map, displaying the ‘correct’ map that all map publishers should follow. The standard map shows the Chinese territorial area extending to include Taiwan and disrupted South China Sea, along with the infamous 11-dash-line. When the announcement was reported by Taiwanese media > netizens of Taiwan derided the map by saying that China is ridiculous and the imagery map is divorced from reality.
• Mapillary announced that anyone can now access their map features directly via an API; filtering and downloading images, detected objects, and map features programmatically.

Upcoming Events

Where	What	When	Country
Taipei	OSM x Wikidata #20	2020-09-07
Lyon	Rencontre mensuelle	2020-09-08
Berlin	147. Berlin-Brandenburg Stammtisch	2020-09-10
Munich	Münchner Treffen	2020-09-10
Zurich	121. OSM Meetup Zurich	2020-09-11
Leoben	Stammtisch Obersteiermark	2020-09-12
San José	National Day of Civic Hacking	2020-09-12
Ashurst	Trek View New Forest Pano Party	2020-09-13
Lüneburg	Lüneburger Mappertreffen	2020-09-15
Salt Lake City / Virtual	OpenStreetMap Utah Map Night	2020-09-15
Kabul / Online	Why OSM and how to Contribute into it on Software Freedom Day 2020	2020-09-18
Nottingham	Nottingham pub meetup	2020-09-22
Düsseldorf	Düsseldorfer OSM-Stammtisch	2020-09-25
Alice	2020 Pista ng Mapa	2020-11-13-2020-11-27

Note: If you like to see your event here, please put it into the calendar. Only data which is there, will appear in weeklyOSM. Please check your event in our public calendar preview and correct it, where appropriate.

This weeklyOSM was produced by AnisKoutsi, Anne Ghisla, MatthiasMatthias, Nakaner, Nordpfeil, NunoMASAzevedo, Polyglot, Rogehm, Sammyhawkrad, TheSwavu, YoViajo, derFred, richter_fn.

Delayed opening

It’s been a while since the opening of an application period was so stressful, but we made it through. While Sage made sure new changes to the codebase were merged as soon as possible, I had the chance to ping communities directly to let them know we would be open for applications soon and invite them to participate in this round. Most of them confirmed their intention to sign up and followed through, one of them signed up without needing a ping, and one of them replied saying they wouldn’t have the time to participate due to COVID-19 constraints.

Sage and I had a few conversations about how to improve the application review workflow (and avoid repetitive strain injury from reviewing thousands of applications). These changes are now live and I’m quite confident they will improve our review practices significantly.

Promotion

In addition to participating in #OutreachyChat, I saw an increase in requests to speak in both public and private events about the program. I received an invitation to become a curator for a week on a project about free and open source software makers, but ultimately decided to schedule my participation to next year to align it with our summer round schedule.

One of our interns requested an interview with an Outreachy organizer, and I’m planning on following up with her this week. Some Brazilian communities have approached me as well, and I’m doing my best to reply their inquiries during the initial application period to encourage Brazil residents to apply.

Valerie Bentivegna is a bio-engineer working and living in Seattle. She is a member of the Seattle pod of 500 Women Scientists and chair of the Communication Working Group of the Marie Curie Alumni Association. You can follow her on her personal blog and on Twitter.

We all know Wikipedia. It’s almost impossible not to. 

For me, from a quick look-up of some fact to prove your point in an argument with friends, to double-checking a chemical structure for schoolwork, or to translate an obscure plant name I can’t think of the English name for; I’ve used Wikipedia consistently for well over a decade.

I’ve always known that Wikipedia was an online encyclopedia than anyone could edit. But I’d never even considered making an edit myself. Until one day in April, I received an email from 500 Women Scientists with the opportunity to attend a 6-week wiki-editing course. I’d already been working from home for a few weeks, with a considerably lower workload than usual, and — to be honest — not quite sure what to do with myself. So, I jumped on the opportunity to learn how to use the skills I already have — hey, I’m a scientist, I’ve been researching and writing and fact-checking for years! — to make Wikipedia a more inclusive place.

500 Women Scientists Wikipedia

About 10 women scientists gathered twice a week to learn how to edit Wikipedia with one main goal: putting more women on Wikipedia. I was saddened, but not surprised, to learn that of all the biographies on Wikipedia, only ~18% are about women. That percentage is ~16% if we only look at academic biographies, and it drops down to ~6.5% for female engineers, my own field. 

One potential reason for this is that a lot of Wikipedia editors are men. And — likely due to implicit bias — they write and edit articles about… other men. Even if the academic world is becoming more inclusive, this isn’t necessarily reflected on the online encyclopedia that everyone uses. 

And that’s a problem. Middle or high schoolers looking to learn more about notable figures in a field of interest and don’t find anyone who looks like them or comes from a similar background, might be turned off from pursuing studies in that field. So that’s where 500 Women Scientists Wikipedia comes in. By increasing representation of women in the academic biography category of Wikipedia, either by improving existing articles or writing new ones (for example through the Women in Red WikiProject, which aims to write articles for “redlinked” women), we could improve representation and therefore make Wikipedia a better and more inclusive resource.

That all sounds good, but how?

Okay, so I knew I wanted to make Wikipedia more inclusive and I knew why, but that didn’t really help me with the “how.” Again, the fact that anyone can edit, doesn’t make me feel comfortable doing so right away! Luckily, the WikiEducators (if that’s the term, the course was organized by Wiki Education, and everything related to Wikipedia seems to have “Wiki” in it!), walked us through the core policies of Wikipedia, the do’s and don’t, and helped us through our first article edits.

Here is a list of things that stuck (but you can find all that is relevant to editing Wikipedia, on — you guessed it! — Wikipedia):

• Statements on Wikipedia must be verifiable, which does not mean they are necessarily true. It just means there’s a sourceable body of work to back up the statement. This feels counterintuitive (shouldn’t we be writing “the truth”?) but it ensures there are reliable sources for everything on Wikipedia.
• Wikipedia is not a place for opinion; articles should reflect a neutral point of view. I did like that this meant according to consensus, as opposed to the journalistic rule of equal time. For example, if 90% of climate researchers are in agreement that climate change is real, that viewpoint should be reflected for 90% of the article.
• To have a biography on Wikipedia, a person must be notable. They have to meet criteria with regard to their academic achievements, prizes won, and impact to merit a presence on the online encyclopedia. In an academic culture where men are typically still more valued than women, this can be another factor for why there are so few biographies about women on Wikipedia.
• The definition of Wikipedia as an “online encyclopedia” is incredibly broad, and apparently it’s easier to define what Wikipedia is not.
• You can contribute to Wikipedia in several different ways, whether it’s writing new content, taking care of layout, correcting spelling and grammar, or making Wikipedia more aesthetically pleasing (just to name a few). 

Making the first edit

The first edit was scary! 

What if I made a mistake? What if I undid someone else’s edit and step on their toes? What if I did something that was inherently anti-Wikipedian?

Wikipedia’s mantra is “Be Bold” — make the change! The beauty of a massively open, crowd-sourced, and peer-reviewed platform is that almost everyone there is willing to help. It’s not seen as a faux-pas to make mistakes, and if you do, someone else will come along and fix it. Accidentally left in a typo? Someone will fix it. Mistakenly got a fact not quite right? Someone will fix it. Change someone’s important edits without noticing? They can come back and undo your change. And Wikipedia keeps track of all the changes in the “history” tab, making the whole editing process transparent and traceable.

Working on the second article was considerably easier. Sure, there are still some really tricky things, like adding images or editing boxes, but overall making edits on Wikipedia is really easy!

“So fix it”

Another Wikipedia Mantra is “So fix it”: if you see something wrong, make it better. 

If you see a lack of representation, write a new article. Make existing articles better (I was surprised to learn about how some articles in the outer corners of Wikipedia are not great). Increasing representation is not just about getting more women biographies on Wikipedia. Black, Indigenous and People of Color academics are more underrepresented on Wikipedia than they are in academia (thanks to the #editWikipedia4BlackLives effort on June 10th and ongoing efforts from the people involved, that will hopefully change), and Pride month brings LBGTQA+ themed “editathons” (sessions where groups of people edit pages together). Wikipedia is a group effort, and together we can all make Wikipedia better: more representative, more inclusive, and more equitable. I myself plan to edit or write one article a week!

Interested in taking a course like the one Valerie took? Visit learn.wikiedu.org to see current course offerings. Another 500 Women Scientists Wiki Scientists course is now enrolling!

You can now use Wikibase data in your Semantic MediaWiki with our new extension "Semantic Wikibase".

Both Wikibase and Semantic MediaWiki are extensions to MediaWiki that allow you to work with structured data within your wiki. They each serve their own use cases and have different strengths and weaknesses.

Our new Semantic Wikibase extension allows you to get the best of both worlds. Data added in the form of Wikibase items and properties is automatically made available in Semantic Mediawiki (SMW). This means you can use it with all SMW functionality, such as embedded visualizations, autocompletion in forms, export in RDF/JSON/CSV/etc and more.

By Andrew Bogott, Senior Site Reliability Engineer, Wikimedia Cloud Services

A couple of years ago I ran across a bug in OpenStack Neutron: I was trying to gather quota information for display in the WMCS tool “Openstack Browser” and got the response “Only admin is authorized to access quotas for another tenant.”

Most API calls in OpenStack are governed by customizable role-based access controls, aka ‘RBAC.’ For some reason Neutron quota weren’t; they were just hard-coded to require admin access. I wasn’t the first person to run into this. There was already a bug entered in the bug tracker and a pending fix.

The pending fix looked pretty good but was a bit out of date.  I tuned it up, resubmitted an updated version, and waited for review. After a bit of back-and-forth, there was a delay while I waited for an answer to a question.  In the meantime, EVERYTHING about RBAC was redesigned in OpenStack, leaving my patch totally broken.  Months later another (in response to prodding from another developer) I finally submitted an updated patch. There were some complications after that with people asking for tests, me adding tests, someone else deciding that maybe the tests were in the wrong place, the CI testing framework breaking, etc. etc. etc., but last week, my tiny patch was finally merged.

OpenStack has fairly rapid release cycles, so this patch will probably be included in an official release in the next couple of months.  This will be OpenStack version ‘Victoria’.  WMCS currently runs OpenStack version ‘Rocky,’ so we will need to upgrade our install 4 times (the release names are alphabetical) before we’re running the fixed version of Neutron in production.

We install OpenStack using upstream Debian packages.  Right now, our Hypervisors are running Debian Stretch; the last version of OpenStack available for Debian Stretch is Rocky.  Before we upgrade from Rocky to Stein we need to upgrade our Virtualization hardware from Stretch to Buster. Typically, we don’t actually ‘upgrade’ hardware; instead, we wipe servers clean and reinstall with a fresh, empty OS. In the case of our Hypervisors, though, there are many (sometimes dozens) of VMs stored locally on the hardware. Wiping the servers would delete our users’ data, so a hypervisor upgrade is a huge, delicate pain in the neck. 

Fortunately, we’re in the process of moving all VMs to distributed storage, after which we can easily transfer VMs here and there to get them safely out of the way of OS upgrades.

To summarize the remaining steps:

• We need to finish moving VMs to Ceph
  • So we can upgrade Hypervisors to Buster
  • So we can upgrade our OpenStack to version Stein
  • So we can upgrade our OpenStack version to Train

• At which point we’ll need to upgrade our OpenStack web interfaces to use version Victoria (Horizon is mercifully backwards-compatible so I don’t have to rebuild it with every release) and upgrade all our custom panels to handle whatever API changes have happened since Rocky…

• And then we can upgrade OpenStack to Ussuri
  • So that we can upgrade Openstack to Victoria

• At which point we can finally fix the Neutron quota issue in OpenStack Browser.

From the perspective of that one little bug, this is a terrible story!  Of course, in reality, this chain of patches and upgrades intersects dozens of other issues and improvements, all stumbling forward together and occasionally knocking each other down or getting in each others’ way in the process. This Neutron issue is just the one lucky bug that got to be on the sidelines to watch it all.

I got to watch it too! The average tech worker keeps a given job for something like 18 months. It’s a genuine privilege to be in one place long enough to watch step after step of a plan come to life and sometimes get to close a bug that I opened years ago.

About this post

Featured image credit: Cirina forda, caterpillar, Lautenschläger et al, CC BY 4.0

During my first week at Newspeak house while explaining Wikidata and Wikibase to some folks on the terrace the topic of Dams came up while discussing an old project that someone had worked on. Back in the day collecting information about Dams would have been quite an effort, compiling a bunch of different data from different sources to try to get a complete worldwide view on the topic. Perhaps it is easier with Wikidata now?

Below is a very brief walkthrough of topic discovery and exploration using various Wikidata features and the SPARQL query service.

A typical known Dam

In order to get an idea of the data space for the topic within Wikidata I start with a Dam that I know about already, the Three Gorges Dam (Q12514). Using this example I can see how Dams are typically described.

Classification

The first thing I notice is that this Dam is an “instance of” “gravity dam”. An “instance of” is represented with the id P31 and a “gravity dam” is represented by the id Q3497167. This is probably a subclass of a wider set. When navigating to gravity dam I see that it is a “subclass of” “dam”. A “sub class of” is represented by the id P279 and “dam” is represented with the id Q12323. This feels like the top level for this ontological tree.

Looking at the talk page for the dam item, I can see some useful links allowing us to dive into the subclasses of “dam”, and also various instances of “dam”.

• List of instances
• List of subclasses
• List of direct subclasses

Properties

Taking another look at the Three Gorges dam item page we see various properties used to describe the dam that might be useful to look at in the dam context:

• image (P18)
• country (P17)
• coordinate location (P625)
• inception (P571)
• drainage basin (P4614)
• discharge (P2225)
• reservoir created (P4661)
• watershed area (P2053)
• annual energy output (P4131)
• installed capacity (P2109)
• … and much more …

The whole set

The best way to get an overview of the whole data set is to use the query service. The dam item talk page already included a link to the query service listing all instances of dam, so we can start there.

This query will list a random set of 1000 “instance of” (P31) or instances of “subclasses of” (P279) the “dam” (Q12323) item, while also providing the English label (name) of the “dam”.

SELECT ?item ?itemLabel WHERE {
  ?item wdt:P31/(wdt:P279)* wd:Q12323 .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en"  }  
}
LIMIT 1000

If we remove the LIMIT from the query we can see that 84215 dams are currently collected in Wikidata. However, sometimes this might not be desired, as the lists can get pretty long.

We can see more information about the dams on this list by expanding the query to look for triples for statements of installed capacity (P2109). Statements are the basic data building blocks in Wikidata connecting a property such as “instance of” with a value such as “dam”. Triples are a representation of this data which is queried by the SPARQL language.

SELECT ?item ?itemLabel ?installedCapacity WHERE {
  ?item wdt:P31/(wdt:P279)* wd:Q12323 .
  ?item wdt:P2109 ?installedCapacity
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en"  }  
}
LIMIT 1000

This will only return dams that have this statement defined (261). In order to still return the complete list, this needs to be added as an OPTIONAL triple.

SELECT ?item ?itemLabel ?installedCapacity WHERE {
  ?item wdt:P31/(wdt:P279)* wd:Q12323 .
  OPTIONAL{ ?item wdt:P2109 ?installedCapacity }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en"  }  
}
LIMIT 1000

More data points can be extracted in much the same way where available.

SELECT ?item ?itemLabel ?installedCapacity ?anualEnergyOutput ?watershedArea WHERE {
  ?item wdt:P31/(wdt:P279)* wd:Q12323 .
  ?item wdt:P2109 ?installedCapacity .
  ?item wdt:P4131 ?anualEnergyOutput .
  ?item wdt:P2053 ?watershedArea .
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en"  }  
}
LIMIT 1000

Other views can also be used for the data. Setting the defaultView option in a comment will make this happen once the query has run.

#defaultView:Map
SELECT ?item ?geo WHERE {
  ?item wdt:P31/wdt:P279* wd:Q12323;
        wdt:P625 ?geo .
}
LIMIT 10000

This map displays 10,000 random dams, and allows you to zoom in, hover and inspect them.

Conclusion

Starting with a topic area to explore and a single known example I have explored the way that Wikidata describes the topic and figured out where the topic fits within the larger tree of concepts. I have also expanded from a single example to a complete data set, visualizing that set on a map.

SPARQL and the query service allow much more than is discussed in this post, such as filtering, alternate data representation and visualization, and much more.

Further reading

• What are triples?
• Check out the Examples button on the query service
• https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service
• https://www.wikidata.org/wiki/Wikidata:SPARQL_tutorial
• SPARQL in 11 minuites (video)

The post How can I get data on all the dams in the world? Use Wikidata appeared first on Addshore.