Distill and Disseminate Generalizable Insights
Sometimes an article you read years ago resurfaces and hits like a bolt of lightning. Andy Matuschak's Ratcheting progress in tools for thought was that bolt for me the last few weeks.
If you've followed along with my journey into applied research, you already know that while I've explored different topics related to the future of computing, I've also reflected on how to do that work. What does it "result" look like in future of computing research? Andy's post answers that question with his typical precision:
the goal of future of computing research is to discover and distill generalizable insights.
Andy suggests this approach:
An idealized cycle of activity might involve something like these key steps:
- Identifying powerful insights about some subject domain or about cognition in general which might be fruitfully systematized
- Building systems which express those insights in their primitives
- Observing serious use of those systems in authentic contexts, and of your theoretical insights refracted through them
- Distilling generalizable insight from those observations which produce new understanding about the subject domain or cognition, and which permit new, better systems to be built
- Disseminating that insight so that others can build on it
My goal is to advance progress for computers as computational medium, so I need to consider this to be the core goal of my work and take seriously this suggested process. These steps should be a guide for the kind of work that I'm doing during my dedicated work blocks.
Let's interrogate each of these steps a bit:
1. Identify a powerful insight
This step largely is about intuition developed through familiarity. Another way to think of this is hypothesis formation: I believe that X is generally true about systems that have Y characteristics in some situations. It's vague, based on gut, but the insight needs to be sharpened enough so that it can guide the next step of building a system that expresses the insight.
2. Build systems that express the insight
I know how to build systems that are "expressions of technological or market insights, rather than of fundamental insights about a subject domain or about cognition." How does one build systems designed for the asking and answering of questions?
Furthermore, I used the plural "systems" intentionally here. It is probable that in order to distill generalizable insights, more than one system will have to be built.
3. Observe serious use of the systems
This implies that the system that is built is actually useful for solving a real problem. In other words, it has to serve an important function with a context of use. People have to want to use it. Building a system that is not merely a research prototype, but is being used in a way that can lead to deriving generalizable insights is challenging and requires a tool-maker mindset.
Beyond usefulness and desirability, "the system has to be shaped in a way which allows you to ask the questions you want to ask." It's not clear to me the characteristics of a system that allow for question-answering. Is it merely analytics instrumentation?
4. Distill generalizable insight
This is the gold nugget. Out of the observation of serious use, some generalizable insight can be distilled. Framing this generalizable insight is a challenge unto itself.
Andy offers a good example from learning science:
Educational design principles are usually heuristic statements of a format such as: “If you want to design intervention X (for the purpose/function Y in context Z), then you are best advised to give that intervention the characteristics A, B, and C (substantive emphasis), and to do that via procedures K, L, and M (procedural emphasis), because of arguments P, Q, and R.”
What might be a good formula for characterizing a generalizable insight for tools for thought?
5. Disseminate the insight
Finally, tell people about it.
This seems to me to be the easiest step, but I'm reminded of the quote attributed to Howard Aiken, himself a pioneer of computing: "Don't worry about people stealing your ideas. If your ideas are any good, you'll have to ram them down people's throats."
Nevertheless, it seems worthwhile to start with the previous four steps and worry about dissemination when there is something worth disseminating.
(The majority of this note also lives in my Research Notebook here.)
ICYMI: Stuff I've worked on
- My Research Notebook got a significant overhaul and expansion. It was like spring cleaning for my decades of notes.
- I did a livestream with Chad Fowler demoing DXOS and we edited it down to this rapid fire 12 minute demo.
Glimpses of the future
The last-ever Strange Loop conference is coming up on September 21-22 in St. Louis, MO and I'm excited to be there. The day after the conference, there will be a local-first software unconference. It's currently full as they search for a larger venue, but if you're interested in local-first, this should be an excellent event to meet and talk.