I just finished the second day of facilitating the Human Computation Roadmap Summit Workshop (through our partner, Sunni Brown, Ink.), in which a highly diverse group of world-class researchers and innovators are exploring the past and prospective impact of human computation to clearly delineate the research areas and activities that will lead directly to the most beneficial national and societal outcomes.
While cleaning up the room after today’s games I had a very interesting discussion with Seth Teicher, who manages Partnerships and Business Development CrowdFlower. I mentioned that I wasn’t a fan of the label “Human Computation” because I think the word “Human” implies there is one person involved in the task. Seth remarked that while he never thought of it this way, he thought my point made sense because at companies like CrowdFlower, the tasks are designed to be solo.
While not perfect, I prefer “Social Computing” because I think it better captures the kinds of tasks that customers use Conteneo’s platforms (Innovation Games® Online, consisting of our Visual Collaboration and Buy a Feature platforms, Knowsy®, and Common Ground for Action) to solve. Specifically, these platforms are designed to solve collaboration problems – which by definition are not solo. (And it doesn’t hurt that Social Computing is the term used in the White House OSTP pcast Report, though it is a real shame that the report mentions the value of cross-agency collaboration without identifying the tools that could be used to achieve this – tools like Conteneo’s platforms!).
Seth pressed me on this, and I’m glad he did, because it helped both of us clarify our respective products and services and the kinds of problems we tackle. Let’s say, for example, that you’re trying to categorize some data. If you can get this task into a form where a single human, working alone, can do what humans do well, then you’ve just created a terrible task for Conteneo’s collaboration platforms and a terrific task for CrowdFlower’s. And that’s the goal of CrowdFlower and platforms like CrowdFlower: design tasks (or micro tasks) that can be performed by humans in isolation, and ideally with low skills or quickly acquirable skills, because isolation in low-skilled tasks leads to scale, and scale leads to low cost.
I asked Seth to consider two different kinds of task, the first of which is certainly faced by CrowdFlower: Prioritizing a product backlog. This is a task in which we want the conversations among stakeholders and customers induced by Buy a Feature precisely because these lead us to not only understand participant priorities, but also understand the reasons behind these priorities and the conditions of acceptance. If you try to get this result through a survey, you’ve certainly reduced the task to n=1, but you’ve lost the rich data provided by Buy a Feature. And if you do reduce the “task” to a survey, then I suspect you wouldn’t use CrowdFlower, because you need much finer-grained control over the participants (market research means market segmentation).
A second kind of Social Computing task I presented to Seth was large distributed team retrospectives. I told him the story about our client who wanted to improve the performance of several hundred developers in India. Again, conceiving this as a human (n=1) task reduces it to a survey – precisely what you don’t want! In Agile at scale, the unit that matters is the team, not the person. And the best way to get feedback from the team is to have the team play a game like Speed Boat online to identify their propellers and anchors. And when we play the game with multiple teams, we generate a dataset that provides deep insight into the opportunities for true organizational improvements.
I could continue, but I think the emerging classification is easy: When the task involves collaboration, especially collaboration between stable social groups, Conteneo’s platforms are the tools of choice. When the task involves perception, categorization, content moderation, and certain forms of content creation in which there is no requirement for collaboration, crowd-sourcing platforms like CrowdFlower are the preferred choice. For example, let’s say I wanted to determine, and potentially improve, the degree of alignment in a team. That’s a problem that Knowsy® is uniquely suited to solve, and one for which I can’t imagine using a crowd sourcing platform to solve, because I don’t collaborate with a crowd).
I’m pretty excited about this insight, because it helps further clarify the distinction between collaborative problem solving and other kinds of problem solving.
I’d like to also give some respect to Pietro Michelucci and the rest of the organizing committee for assembling a stellar group of people who have eagerly tackled a bunch of complex tasks. For example, today we played Missing Pieces, My Worst Nightmare while building a research roadmap that captured how research areas and activities leads to better understanding of solution components and how these solution components lead to robust and scalable solutions. The seven teams then presented their work to Tom Kalil, the Deputy Director for Policy for the White House Office of Science and Technology Policy and Senior Advisor for Science, Technology and Innovation for the National Economic Council and a renowned expert in technology and innovation. His insights and feedback, and his ability to make recommendations and connections to other work was remarkable.