AI Summit_Sept. 13 2024

Inference Phase

Why is this a concern?

Group

Risk

Indicator

Prompt data might be stored or later used for other purposes like model evaluation and retraining. These types of data must be reviewed with respect to privacy laws and regulations. Without proper data storage and usage business entities could ³ Q Q Q other legal consequences. Prompt data might be stored or later used for other purposes like model evaluation and retraining. These types of data must be reviewed with respect to IP laws and regulations. Without proper data storage and usage business entities could face ³ Q Q Q legal consequences.

New

Privacy

Personal information in prompt: Disclosing Personal Information or Sensitive Personal Information as a part of prompt sent to the model.

Intellectual Property

IP information in prompt: Disclosing copyright information or other IP information as a part of the prompt sent to the model.

New

) ³ Q ³ )0 N ! Q Y ³ information might be unintentionally collected and stored.

New

# ³ P ) ³ prompt sent to the model.

% Q ³ attacker. If the output results are not properly accounted Q ³ Q Q disruption to operations, and other legal consequences.

! ³

Robustness Evasion attack: attempt to make a model output incorrect results by perturbing the data sent to the trained model.

Depending on the content revealed, business entities could ³ Q Q Q other legal consequences.

New

Prompt-based attacks: Adversarial attacks such as prompt injection (attempt to force a model to produce unexpected output), prompt leaking (attempts to extract a model’s system prompt), jailbreaking (attempts to break through the guardrails established in the model), and prompt priming (attempt to force a model to produce an output aligned to the prompt).

11

Foundation models: Opportunities, risks and mitigations | February 2024

AI Roundtable Page 685