Managing AI agents means managing judgment
From the workshop

Managing AI agents means managing judgment

Adam Kalisz
EN#judgment#decision

⚖️ Judging what your and others' agents produced all day?

🖋️ You are running a Distributed Denial of Service attack against your mental capacity. You have become the humanities teacher giving out assignments and, as a result, ending up with tall stacks of essays to review.

Okay, so you come to the conclusion that your scaffolding and safety nets are strong enough that you only need to review a fraction of the code to keep your sanity while moving forward. You are using your hard skills less and doing more project management and quality control. You don't recognize parts of the system, and you are unsure what certain components are for.

👉 We are facing a shift where the work we do is as much about the work itself as it is about the person doing the work. To understand why this is happening, we have to look at the properties of the work at hand and the attributes of the person doing the work.

Is the work:

  • necessary,
  • physically demanding,
  • psychologically or emotionally demanding,
  • cognitively demanding,
  • varied, creative, or repetitive?

Is the person:

  • responsible or trustworthy,
  • experienced or skilled,
  • empathetic or respectful,
  • likeable or popular?

🏋️ We have automated a lot of the necessary, physically demanding, and repetitive work. Thanks to appliances and machines, we have made many types of work significantly less physically demanding. Yet we still have all the soft edge cases, which demand judgment. The ratio of #judgment, of having to make a #decision, to the rest of the work increases sharply, especially when we look at the time spent.

🤖 We are becoming observers, assistants, plumbers, and managers for the specialist robots doing the tasks. We just create the working conditions, fix problems, find work, do maintenance, and generally tend to the machines.

But machines are not really taking our psychologically and emotionally demanding jobs either, except perhaps for some low-stakes call center work.

Just imagine calling the police or the fire department and having the nagging feeling that you are talking to a bot. We also know how it can end when people mistake bots for psychologists and psychiatrists.

Coming up with work, e.g., thinking about what will be done, by whom, approximately how, and whether it is necessary, is also work in itself.

🙋 Yes, this is more about the person doing the work than the work itself. A person knows when to push back and inquire to find out whether the work is actually necessary, or necessary in its current form. This is because good judgment and taste are, at some level, still very human qualities.