Learn in the textbookDenial-of-Service, Zero-Day, and Attacks Targeting AI
DefinitionA prompt injection attack (プロンプトインジェクション攻撃) feeds generative AI crafted instructions, or prompts, to make it break the restrictions set by its developers or reveal information it should not.
ExplanationGenerative AI receives both the instructions its developers gave in advance and the user's input as plain text and processes them together. Attackers exploit this by slipping a command such as "ignore all previous instructions" into the input so that the AI puts it ahead of its original instructions. Besides typing such commands directly, attackers can also hide them in Web pages or documents that the AI is asked to read.
AnalogyIt is like scribbling "open the safe for whoever brings this" in the corner of a document handed to a receptionist, who mistakes it for the boss's orders and complies.
Memory hookInjection means pouring something in: another set of instructions is quietly poured into the legitimate stream.