mastodon.world is one of the many independent Mastodon servers you can use to participate in the fediverse.
Generic Mastodon server for anyone to use.

Server stats:

9.3K
active users

#adverseselection

0 posts0 participants0 posts today
Replied in thread

@ZaneSelvans
Climate change will break the insurance industry before it breaks everything else.

Actuaries are the world’s oldest data scientists.

The math doesn’t lie.

They are paid to avoid .

When the risks manifest everywhere, on everything, all the time, actuarial models collapse.

"One million properties are projected to become chronically flooded: properties that today fund nearly 30% of local revenues for more than half of the state’s municipalities, according to a new study conducted by researchers at Cornell and Florida State Universities."

wmfe.org/environment/2023-10-1

WMFE · New study projects sea level rise to drain Florida’s financial futureBy Molly Duerig

Of Models and Tin Men
arxiv.org/abs/2307.11137
An ambitious research agenda:
"In a #PrincipalAgentProblem, conflict arises because of information asymmetry together with inherent misalignment between the utility of the agent and the principal… argue the assumptions underlying principal-agent problems are
crucial to capturing the essence of safety problems involving pre-trained AI models in real-world situations."
#llm #AIEthics #MoralHazard #AdverseSelection #economics

arXiv.orgOf Models and Tin Men -- a behavioural economics study of principal-agent problems in AI alignment using large-language modelsAI Alignment is often presented as an interaction between a single designer and an artificial agent in which the designer attempts to ensure the agent's behavior is consistent with its purpose, and risks arise solely because of conflicts caused by inadvertent misalignment between the utility function intended by the designer and the resulting internal utility function of the agent. With the advent of agents instantiated with large-language models (LLMs), which are typically pre-trained, we argue this does not capture the essential aspects of AI safety because in the real world there is not a one-to-one correspondence between designer and agent, and the many agents, both artificial and human, have heterogeneous values. Therefore, there is an economic aspect to AI safety and the principal-agent problem is likely to arise. In a principal-agent problem conflict arises because of information asymmetry together with inherent misalignment between the utility of the agent and its principal, and this inherent misalignment cannot be overcome by coercing the agent into adopting a desired utility function through training. We argue the assumptions underlying principal-agent problems are crucial to capturing the essence of safety problems involving pre-trained AI models in real-world situations. Taking an empirical approach to AI safety, we investigate how GPT models respond in principal-agent conflicts. We find that agents based on both GPT-3.5 and GPT-4 override their principal's objectives in a simple online shopping task, showing clear evidence of principal-agent conflict. Surprisingly, the earlier GPT-3.5 model exhibits more nuanced behaviour in response to changes in information asymmetry, whereas the later GPT-4 model is more rigid in adhering to its prior alignment. Our results highlight the importance of incorporating principles from economics into the alignment process.