Prompt injection is when malicious user input manipulates the model into ignoring its system prompt instructions. Defences: clearly delimit untrusted input in the prompt (XML tags, explicit labelling), instruct the model to treat user content as data not commands, validate outputs, and use least-privilege tool access.
Audit training data and outputs for demographic bias, test across diverse user groups, use bias-detection tools (Fairlearn, AI Fairness 360), document model cards with known limitations, implement human review for high-impact decisions, and establish a feedback mechanism for affected users to report harm.
The EU AI Act (risk-tiered regulation, high-risk systems require conformity assessments), GDPR/Privacy laws for any personal data used in training or inference, US sector-specific guidance (FTC, FDA for medical AI), and Australia’s voluntary AI Ethics Framework. Requirements vary by jurisdiction and use case — legal review is essential for high-risk deployments.
For application developers: implement output filtering and content moderation, add human-in-the-loop for irreversible or high-stakes actions, apply least-privilege principles to agent tool access, log all decisions for auditability, define clear escalation paths, and stay updated on your model provider’s safety guidelines and model cards.
