💭 on natural many llm chat (as opposed to long-form turn taking).

At a 10 person public discussion, turn taking fails terribly. A challenges D. B gives evidence against A. C talks about F. By the time A is allowed to contest B's comment, it's one of many simultaneous conversations, and now A must also comment on the other present conversations. Every message is overburdened and the chat history is full of such walls of text.

Two challenges to fix this. Cost is one since shorter messages likely means more total messages (though ideally not everyone needs to weigh in on every claim). I'm hoping a strong llm can make a plan / playbook which can be performed by a cheaper llm. I fully anticipate being disappointed.

The other is knowing when to speak, since of course an llm is not "hearing" the chat until it is prompted to respond and thus accruing cost. A cheap llm could represent each actor, though frankly I don't have much faith in cheap classified llms and imagine by default everyone would want to comment on everything and tuning it could be painful. Another option is an invisible moderator who chooses who should speak next possibly with some kind of weighted queue. Each actor can list what subjects they might want to comment on, and the moderator could enqueue anyone directly mentioned or who appears relevant given their list. That still would necessarily lead to more opportunities to talk than actual messages (presuming most messages could potentially be responded to by multiple others) so I suppose I have to hope the cheap llms are cheap enough and can be tuned to shut up when they have nothing important or new to say.