Policy "dogfooding" / scientific method in digital policy making

Photo of a bowl of multi-coloured dog food pellets


the practice of using one's own products or services

Something Must Be Done. By someone else.

I read a lot of policy papers and research papers. Many of them examine aspects of online services, and usually end with a plea that Something Must Be Done in support of the object of the paper.

The "online harms" debate is a prime example of this, with lots of people with ideas of What Must Be Done, but far less critical thinking on the specifics of the harms to be tackled, or thinking beyond "tech is the solution", or actual practical implementations of the proposals being advanced.

One of the more curious “papers” I’ve read recently - "Playful by design: Free play in a digital world" - argues that Zoom et al should be doing more to support children using it for “free play”.

(Important note: although it was this paper which compelled me to write down my thoughts on this, I'm not criticising this particular paper. My post is thematic, rather than specific.)

The report says:

“society should hold out high expectations for the quality of children’s play across all environments.”

The argument is that (all?)) online services should be "playful by design".

This stretches the argument that all online services should be safe by default for unsupervised children even further, positing that online services - even business conferencing services? - should redesign themselves to support children’s play.

The paper says:

So while Zoom supports some qualities of free play (social, imaginative, voluntary, diverse and open-ended), we heard little of its potential for being emotionally resonant, immersive, or enabling risk-taking or a sense of achievement. And it has limitations in terms of enabling intrinsically motivated play and safety.”

I'm not surprised. It's a business video conferencing tool.

Criticising it for its "limitations in terms of enabling intrinsically motivated play and safety" is rather like complaining that an office into which a parent brings their child while they work lacks the facilities of a creche, nursery, or soft play centre.

The thesis seems to be:

  • Zoom exists
  • Some children use Zoom, even though not for the purpose for which Zoom exists
  • Children need to play
  • Zoom should redesign its system to enable children to play

This particular paper — there is a promise of a "full report" to come — does not comment on what this means in practice, or the economics or viability of the proposal on a business video conferencing tool's commercial proposition.

The next paper seems to promise at least some attention to this:

Next we will identify the digital architecture that enables the qualities of free play to flourish. By mapping the design features that enable or inhibit free play, we hope to get one step closer to a digital world that is playful by design.

I'm mindful, though, that it stops short of promising to address the fundamental commercial / business questions of shifting from a business video conferencing service to a digital playground.

I'm doubtful - but happy to be proven wrong - that the next document will present a costed analysis with specific functional and non-functional requirements.

Why? Because without actually building an experimental solution, this would be nigh on impossible.

Fewer graphics, more dogfooding / scientic method in digital policy making

I expect I’m just a grump, but rather than spending money on a rather brightly coloured PDF, why not do the very thing you’re suggesting that others should do, and build an online service to support children’s free play?

Build it, make it safe and fun for children, and run it. Get children using it — build a user base!

In essence, injecting scientific method into policy debates.

After all, it's one thing to come up with some proposals, and quite another to assess their viability, and their impact, in a practical sense. Anyone can draw a plane, but making one that actually flies is a different kettle of fish entirely.

Fortunately, for policy proposals in the digital space, the barrier to entry is a lot, lot lower than "building your own plane".

For example, if you're saying that a video conference service should be "playful by design", rather than dunking on Zoom, why not grab the source code to Jitsi Meet as a starting point, and build the service you are envisaging?

It’s open source, and you can pay devs if you can’t code it yourself, and you're already building on a massively successful, extensible platform.

Build it to hit all 12 points for free play for children, and launch it.

Test if the report's thesis is viable.

See how is it used, or abused.

Check if your requirements, functional or non-functional, are feasible for implementation in practice. Tweak them if not, adjust the code, and try again.

Is the proposal realistic? What is the user experience like? How much does doing all this cost, and are the responsible adults of the target users willing to pay for it?

And when you've built it, release it under a Free / open source licence, so that others can implement it, extend it, incorporate it, run it, so that it can be adopted widely.

Why would you not dogfood your own policy recommendations?

If you are professionally involved in policy making, and espouse a thesis in favour of particular requirements or a particular solution, but you are not running up even a bare bones proof of concept system and testing it in a real world scenario, why not?

Would it not make your policy influencing more effective if you could show actual, real world, impact?

If you could demonstrate that your thesis is not pie in the sky, but is implementable? Especially if part of your thesis (not evident in this particular report, to be clear) is that organisations just need to make their staff "nerd harder" to solve seemingly intractable societal problems via tech solutionism.

Yes, this would require more than just producing reports, and would necessitate bringing "do-ers" – coders, and others who can turn ideas into reality — into the policy arena.

And, in doing so — by experimenting in code, with actual users, rather than on paper, and formulating and building and tweaking and adjusting policy based on your experimental findings - it strikes me that digital policy making could improve dramatically.

Image credit

Dog Food 2, by Sh4rp_i. Original image. Licensed under CC BY 2.0.

Author: neil

I'm Neil. By day, I run a law firm, decoded.legal, giving advice on Internet, telecoms, and tech law. This is my personal blog, so will be mostly about tech stuff, cycling, and other hobbies.