Cryptography and Security
- [1] arXiv:2406.05134 [pdf, ps, other]
-
Title: oBAKE: an Online Biometric-Authenticated Key Exchange ProtocolComments: 4 pages, 2 figuresSubjects: Cryptography and Security (cs.CR)
In this writing, we introduce a novel biometric-authenticated key exchange protocol that allows secure and privacy-preserving key establishment between a stateless biometric sensing system and a "smart" user token that possesses biometric templates of the user.
The protocol yields a shared secret incorporating random nonce from both parties when they positively authenticate each other. Mutual positive authentication here is defined as when the feature vector calculated from the sensor data captured by the biometric sensing system only differs from the feature vector stored as the biometric template within the user token by less than a predefined threshold. The parties exchange only randomized data and cryptographically derived verifiers; no significant information regarding the vectors is ever exchanged. The protocol essentially utilizes the BBKDF scheme for feature vector matching, and as a result, the threshold is compared per component of the two vectors to be matched. This fact makes it straightforward to employ multiple biometric modalities.
The protocol also allows online authentication where the biometric sensing system can potentially send multiple queries derived from different sensor data samples, in one or more rounds. The protocol is designed in such a way that the user token can very efficiently answer a multitude of such queries. This makes the protocol especially suitable for interactive systems while posing a minimal computational burden on the user token. The biometric sensing system can be made stateless, i.e. user registration in advance is not required.
Furthermore, the protocol is bidirectionally privacy-preserving in the sense that unless mutual authentication is achieved first, neither the biometric sensing system, nor the user token can gain useful information, respectively regarding the biometric template, or sensor-data-derived feature vectors. - [2] arXiv:2406.05137 [pdf, ps, other]
-
Title: Multi-sensor Intrusion Detection SystemSubjects: Cryptography and Security (cs.CR); Systems and Control (eess.SY)
Security, defined as protection against external threats, is a critical concern for homes and offices. Intrusion, characterized by unauthorized access, presents a significant challenge to maintaining security. This research aims to address this issue by designing and implementing an automated intrusion detection system utilizing a combination of sensors and communication technologies.
The research introduced an automated intrusion detection system for homes and offices, combining sensors such as a PIR sensor for detecting unauthorized motion, magnetic switches for unauthorized entry detection, and a GSM module for notifying property owners. Employing the ATmega328P microcontroller, sensor data is analysed to generate early intrusion alerts, prompting phone call notifications via the GSM module. Practical implementation involved breadboarding, soldering, and rigorous testing, ensuring proper functionality under real-world conditions. The implemented intrusion detection system effectively utilizes magnetic switches and a Passive Infrared (PIR) sensor to detect unauthorized entry and motion within the premises, respectively. Upon detection, the system promptly analyses the situation and alerts the property owner via phone call, enabling swift response measures. This real-time notification system enhances proactive security management, minimizing the risk of further intrusion and ensuring the safety of the property. The multi-sensor intrusion detection system, incorporating PIR sensors, magnetic switches, and a GSM-based phone call gateway, effectively alerts property owners of unauthorized intrusions in real-time. Demonstrating its efficacy through rigorous testing, the system offers enhanced security for both residential and commercial environments. - [3] arXiv:2406.05151 [pdf, ps, other]
-
Title: CredSec: A Blockchain-based Secure Credential Management System for University AdoptionComments: 10 pages, 7 figures, 3 tablesSubjects: Cryptography and Security (cs.CR)
University education play a critical role in shaping intellectual and professional development of the individuals and contribute significantly to the advancement of knowledge and society. Generally, university authority has a direct control of students result making and stores the credential in their local dedicated server. So, there is chance to alter the credential and also have a very high possibility to encounter various threats and different security attacks. To resolve these, we propose a blockchain based secure credential management system (BCMS) for efficiently storing, managing and recovering credential without involving the university authority. The proposed BCMS incorporates a modified two factor encryption (m2FE) technique, a combination of RSA cryptosystem and a DNA encoding to ensure credential privacy and an enhanced authentication scheme for teachers and students. Besides, to reduce size of the cipher credential and its conversion time, we use character to integer (C2I) table instead of ASCII table. Finally, the experimental result and analysis of the BCMS illustrate the effectiveness over state of the art works.
- [4] arXiv:2406.05310 [pdf, ps, html, other]
-
Title: COOKIEGUARD: Characterizing and Isolating the First-Party Cookie JarSubjects: Cryptography and Security (cs.CR)
As third-party cookies are going away, first-party cookies are increasingly being used for tracking. Prior research has shown that third-party scripts write (or \textit{ghost-write}) first-party cookies in the browser's cookie jar because they are included in the website's main frame. What is more is that a third-party script is able to access all first-party cookies, both the actual first-party cookies as well as the ghost-written first-party cookies by different third-party scripts. Existing isolation mechanisms in the web browser such as SOP and CSP are not designed to address this lack of isolation between first-party cookies written by different third-parties. We conduct a comprehensive analysis of cross-domain first-party cookie retrieval, exfiltration, and modification on top-10K websites. Most notably, we find 18\% and 4\% of the first-party cookies are exfiltrated and overwritten, respectively, by cross-domain third-party scripts. We propose \name to introduce isolation between first-party cookies set by different third-party scripts in the main frame. To this end, \name intercepts cookie get and set operations between third-party scripts and the browser's cookie jar to enforce strict isolation between first-party cookies set by different third-party domains. Our evaluation of \name shows that it effectively blocks all cross-domain cookie read/write operations to provide a fully isolated cookie jar. While it generally does not impact appearance, navigation, or other website functionality, the strict isolation policy disrupts Single Sign-On (SSO) on just 11\% of websites that rely on first-party cookies for session management. Our work demonstrates the feasibility of isolating first-party cookies.
- [5] arXiv:2406.05362 [pdf, ps, html, other]
-
Title: RAPID: Robust APT Detection and Investigation Using Context-Aware Deep LearningSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Advanced persistent threats (APTs) pose significant challenges for organizations, leading to data breaches, financial losses, and reputational damage. Existing provenance-based approaches for APT detection often struggle with high false positive rates, a lack of interpretability, and an inability to adapt to evolving system behavior. We introduce RAPID, a novel deep learning-based method for robust APT detection and investigation, leveraging context-aware anomaly detection and alert tracing. By utilizing self-supervised sequence learning and iteratively learned embeddings, our approach effectively adapts to dynamic system behavior. The use of provenance tracing both enriches the alerts and enhances the detection capabilities of our approach. Our extensive evaluation demonstrates RAPID's effectiveness and computational efficiency in real-world scenarios. In addition, RAPID achieves higher precision and recall than state-of-the-art methods, significantly reducing false positives. RAPID integrates contextual information and facilitates a smooth transition from detection to investigation, providing security teams with detailed insights to efficiently address APT threats.
- [6] arXiv:2406.05364 [pdf, ps, html, other]
-
Title: Is On-Device AI Broken and Exploitable? Assessing the Trust and Ethics in Small Language ModelsComments: 26 pages, 31 figures and 5 tablesSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
In this paper, we present a very first study to investigate trust and ethical implications of on-device artificial intelligence (AI), focusing on ''small'' language models (SLMs) amenable for personal devices like smartphones. While on-device SLMs promise enhanced privacy, reduced latency, and improved user experience compared to cloud-based services, we posit that they might also introduce significant challenges and vulnerabilities compared to on-server counterparts.
As part of our trust assessment study, we conduct a systematic evaluation of the state-of-the-art on-devices SLMs, contrasted to their on-server counterparts, based on a well-established trustworthiness measurement framework. Our results show on-device SLMs to be (statistically) significantly less trustworthy, specifically demonstrating more stereotypical, unfair and privacy-breaching behavior. Informed by these findings, we then perform our ethics assessment study by inferring whether SLMs would provide responses to potentially unethical vanilla prompts, collated from prior jailbreaking and prompt engineering studies and other sources. Strikingly, the on-device SLMs did answer valid responses to these prompts, which ideally should be rejected. Even more seriously, the on-device SLMs responded with valid answers without any filters and without the need for any jailbreaking or prompt engineering. These responses can be abused for various harmful and unethical scenarios including: societal harm, illegal activities, hate, self-harm, exploitable phishing content and exploitable code, all of which indicates the high vulnerability and exploitability of these on-device SLMs. Overall, our findings highlight gaping vulnerabilities in state-of-the-art on-device AI which seem to stem from resource constraints faced by these models and which may make typical defenses fundamentally challenging to be deployed in these environments. - [7] arXiv:2406.05403 [pdf, ps, html, other]
-
Title: SemPat: Using Hyperproperty-based Semantic Analysis to Generate Microarchitectural Attack PatternsComments: 16 pages, 14 figures, under reviewSubjects: Cryptography and Security (cs.CR); Hardware Architecture (cs.AR)
Microarchitectural security verification of software has seen the emergence of two broad classes of approaches. The first is based on semantic security properties (e.g., non-interference) which are verified for a given program and a specified abstract model of the hardware microarchitecture. The second is based on attack patterns, which, if found in a program execution, indicates the presence of an exploit. While the former uses a formal specification that can capture several gadget variants targeting the same vulnerability, it is limited by the scalability of verification. Patterns, while more scalable, must be currently constructed manually, as they are narrower in scope and sensitive to gadget-specific structure.
This work develops a technique that, given a non-interference-based semantic security hyperproperty, automatically generates attack patterns up to a certain complexity parameter (called the skeleton size). Thus, we combine the advantages of both approaches: security can be specified by a hyperproperty that uniformly captures several gadget variants, while automatically generated patterns can be used for scalable verification. We implement our approach in a tool and demonstrate the ability to generate new patterns, (e.g., for SpectreV1, SpectreV4) and improved scalability using the generated patterns over hyperproperty-based verification. - [8] arXiv:2406.05443 [pdf, ps, other]
-
Title: Novel Approach to Intrusion Detection: Introducing GAN-MSCNN-BILSTM with LIME PredictionsJournal-ref: Data and Metadata, 2023 Dec. 28Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Networking and Internet Architecture (cs.NI)
This paper introduces an innovative intrusion detection system that harnesses Generative Adversarial Networks (GANs), Multi-Scale Convolutional Neural Networks (MSCNNs), and Bidirectional Long Short-Term Memory (BiLSTM) networks, supplemented by Local Interpretable Model-Agnostic Explanations (LIME) for interpretability. Employing a GAN, the system generates realistic network traffic data, encompassing both normal and attack patterns. This synthesized data is then fed into an MSCNN-BiLSTM architecture for intrusion detection. The MSCNN layer extracts features from the network traffic data at different scales, while the BiLSTM layer captures temporal dependencies within the traffic sequences. Integration of LIME allows for explaining the model's decisions. Evaluation on the Hogzilla dataset, a standard benchmark, showcases an impressive accuracy of 99.16\% for multi-class classification and 99.10\% for binary classification, while ensuring interpretability through LIME. This fusion of deep learning and interpretability presents a promising avenue for enhancing intrusion detection systems by improving transparency and decision support in network security.
- [9] arXiv:2406.05451 [pdf, ps, other]
-
Title: PrivacyCube: Data Physicalization for Enhancing Privacy Awareness in IoTSubjects: Cryptography and Security (cs.CR); Emerging Technologies (cs.ET)
People are increasingly bringing Internet of Things (IoT) devices into their homes without understanding how their data is gathered, processed, and used. We describe PrivacyCube, a novel data physicalization designed to increase privacy awareness within smart home environments. PrivacyCube visualizes IoT data consumption by displaying privacy-related notices. PrivacyCube aims to assist smart home occupants to (i) understand their data privacy better and (ii) have conversations around data management practices of IoT devices used within their homes. Using PrivacyCube, households can learn and make informed privacy decisions collectively. To evaluate PrivacyCube, we used multiple research methods throughout the different stages of design. We first conducted a focus group study in two stages with six participants to compare PrivacyCube to text and state-of-the-art privacy policies. We then deployed PrivacyCube in a 14-day-long field study with eight households. Our results show that PrivacyCube helps home occupants comprehend IoT privacy better with significantly increased privacy awareness at p < .05 (p=0.00041, t= -5.57). Participants preferred PrivacyCube over text privacy policies because it was comprehensive and easier to use. PrivacyCube and Privacy Label, a state-of-the-art approach, both received positive reviews from participants, with PrivacyCube being preferred for its interactivity and ability to encourage conversations. PrivacyCube was also considered by home occupants as a piece of home furniture, encouraging them to socialize and discuss IoT privacy implications using this device.
- [10] arXiv:2406.05459 [pdf, ps, other]
-
Title: PriviFy: Designing Tangible Interfaces for Configuring IoT Privacy PreferencesSubjects: Cryptography and Security (cs.CR); Emerging Technologies (cs.ET)
The Internet of Things (IoT) devices, such as smart speakers can collect sensitive user data, necessitating the need for users to manage their privacy preferences. However, configuring these preferences presents users with multiple challenges. Existing privacy controls often lack transparency, are hard to understand, and do not provide meaningful choices. On top of that, users struggle to locate privacy settings due to multiple menus or confusing labeling, which discourages them from using these controls. We introduce PriviFy (Privacy Simplify-er), a novel and user-friendly tangible interface that can simplify the configuration of smart devices privacy settings. PriviFy is designed to propose an enhancement to existing hardware by integrating additional features that improve privacy management. We envision that positive feedback and user experiences from our study will inspire consumer product developers and smart device manufacturers to incorporate the useful design elements we have identified. Using fidelity prototyping, we iteratively designed PriviFy prototype with 20 participants to include interactive features such as knobs, buttons, lights, and notifications that allow users to configure their data privacy preferences and receive confirmation of their choices. We further evaluated PriviFy high-fidelity prototype with 20 more participants. Our results show that PriviFy helps simplify the complexity of privacy preferences configuration with a significant usability score at p < .05 (P = 0.000000017, t = -8.8639). PriviFy successfully met users privacy needs and enabled them to regain control over their data. We conclude by recommending the importance of designing specific privacy configuration options.
- [11] arXiv:2406.05472 [pdf, ps, html, other]
-
Title: A Novel Generative AI-Based Framework for Anomaly Detection in Multicast Messages in Smart Grid CommunicationsComments: 10 pages, 10 figures, Submitted to IEEE Transactions on Information Forensics and SecuritySubjects: Cryptography and Security (cs.CR); Systems and Control (eess.SY)
Cybersecurity breaches in digital substations can pose significant challenges to the stability and reliability of power system operations. To address these challenges, defense and mitigation techniques are required. Identifying and detecting anomalies in information and communication technology (ICT) is crucial to ensure secure device interactions within digital substations. This paper proposes a task-oriented dialogue (ToD) system for anomaly detection (AD) in datasets of multicast messages e.g., generic object oriented substation event (GOOSE) and sampled value (SV) in digital substations using large language models (LLMs). This model has a lower potential error and better scalability and adaptability than a process that considers the cybersecurity guidelines recommended by humans, known as the human-in-the-loop (HITL) process. Also, this methodology significantly reduces the effort required when addressing new cyber threats or anomalies compared with machine learning (ML) techniques, since it leaves the models complexity and precision unaffected and offers a faster implementation. These findings present a comparative assessment, conducted utilizing standard and advanced performance evaluation metrics for the proposed AD framework and the HITL process. To generate and extract datasets of IEC 61850 communications, a hardware-in-the-loop (HIL) testbed was employed.
- [12] arXiv:2406.05498 [pdf, ps, html, other]
-
Title: SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical MannerXunguang Wang, Daoyuan Wu, Zhenlan Ji, Zongjie Li, Pingchuan Ma, Shuai Wang, Yingjiu Li, Yang Liu, Ning Liu, Juergen RahmelComments: This paper completes its earlier vision paper, available at arXiv:2402.15727Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs) and has evolved into four major categories: optimization-based attacks such as Greedy Coordinate Gradient (GCG), jailbreak template-based attacks such as "Do-Anything-Now", advanced indirect attacks like DrAttack, and multilingual jailbreaks. However, delivering a practical jailbreak defense is challenging because it needs to not only handle all the above jailbreak attacks but also incur negligible delay to user prompts, as well as be compatible with both open-source and closed-source LLMs.
Inspired by how the traditional security concept of shadow stacks defends against memory overflow attacks, this paper introduces a generic LLM jailbreak defense framework called SelfDefend, which establishes a shadow LLM defense instance to concurrently protect the target LLM instance in the normal stack and collaborate with it for checkpoint-based access control. The effectiveness of SelfDefend builds upon our observation that existing LLMs (both target and defense LLMs) have the capability to identify harmful prompts or intentions in user queries, which we empirically validate using the commonly used GPT-3.5/4 models across all major jailbreak attacks. Our measurements show that SelfDefend enables GPT-3.5 to suppress the attack success rate (ASR) by 8.97-95.74% (average: 60%) and GPT-4 by even 36.36-100% (average: 83%), while incurring negligible effects on normal queries. To further improve the defense's robustness and minimize costs, we employ a data distillation approach to tune dedicated open-source defense models. These models outperform four SOTA defenses and match the performance of GPT-4-based SelfDefend, with significantly lower extra delays. We also empirically show that the tuned models are robust to targeted GCG and prompt injection attacks. - [13] arXiv:2406.05517 [pdf, ps, other]
-
Title: Blockchain Integrated Federated Learning in Edge-Fog-Cloud Systems for IoT based Healthcare Applications A SurveySubjects: Cryptography and Security (cs.CR)
Modern Internet of Things (IoT) applications generate enormous amounts of data, making data-driven machine learning essential for developing precise and reliable statistical models. However, data is often stored in silos, and strict user-privacy legislation complicates data utilization, limiting machine learning's potential in traditional centralized paradigms due to diverse data probability distributions and lack of personalization. Federated learning, a new distributed paradigm, supports collaborative learning while preserving privacy, making it ideal for IoT applications. By employing cryptographic techniques, IoT systems can securely store and transmit data, ensuring consistency. The integration of federated learning and blockchain is particularly advantageous for handling sensitive data, such as in healthcare. Despite the potential of these technologies, a comprehensive examination of their integration in edge-fog-cloud-based IoT computing systems and healthcare applications is needed. This survey article explores the architecture, structure, functions, and characteristics of federated learning and blockchain, their applications in various computing paradigms, and evaluates their implementations in healthcare.
- [14] arXiv:2406.05590 [pdf, ps, html, other]
-
Title: NYU CTF Dataset: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive SecurityMinghao Shao, Sofija Jancheska, Meet Udeshi, Brendan Dolan-Gavitt, Haoran Xi, Kimberly Milner, Boyuan Chen, Max Yin, Siddharth Garg, Prashanth Krishnamurthy, Farshad Khorrami, Ramesh Karri, Muhammad ShafiqueSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computers and Society (cs.CY); Machine Learning (cs.LG)
Large Language Models (LLMs) are being deployed across various domains today. However, their capacity to solve Capture the Flag (CTF) challenges in cybersecurity has not been thoroughly evaluated. To address this, we develop a novel method to assess LLMs in solving CTF challenges by creating a scalable, open-source benchmark database specifically designed for these applications. This database includes metadata for LLM testing and adaptive learning, compiling a diverse range of CTF challenges from popular competitions. Utilizing the advanced function calling capabilities of LLMs, we build a fully automated system with an enhanced workflow and support for external tool calls. Our benchmark dataset and automated framework allow us to evaluate the performance of five LLMs, encompassing both black-box and open-source models. This work lays the foundation for future research into improving the efficiency of LLMs in interactive cybersecurity tasks and automated task planning. By providing a specialized dataset, our project offers an ideal platform for developing, testing, and refining LLM-based approaches to vulnerability detection and resolution. Evaluating LLMs on these challenges and comparing with human performance yields insights into their potential for AI-driven cybersecurity solutions to perform real-world threat management. We make our dataset open source to public this https URL along with our playground automated framework this https URL.
- [15] arXiv:2406.05870 [pdf, ps, html, other]
-
Title: Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker DocumentsSubjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)
Retrieval-augmented generation (RAG) systems respond to queries by retrieving relevant documents from a knowledge database, then generating an answer by applying an LLM to the retrieved documents.
We demonstrate that RAG systems that operate on databases with potentially untrusted content are vulnerable to a new class of denial-of-service attacks we call jamming. An adversary can add a single ``blocker'' document to the database that will be retrieved in response to a specific query and, furthermore, result in the RAG system not answering the query - ostensibly because it lacks the information or because the answer is unsafe.
We describe and analyze several methods for generating blocker documents, including a new method based on black-box optimization that does not require the adversary to know the embedding or LLM used by the target RAG system, nor access to an auxiliary LLM to generate blocker documents. We measure the efficacy of the considered methods against several LLMs and embeddings, and demonstrate that the existing safety metrics for LLMs do not capture their vulnerability to jamming. We then discuss defenses against blocker documents. - [16] arXiv:2406.05874 [pdf, ps, html, other]
-
Title: Stealthy Targeted Backdoor Attacks against Image CaptioningSubjects: Cryptography and Security (cs.CR)
In recent years, there has been an explosive growth in multimodal learning. Image captioning, a classical multimodal task, has demonstrated promising applications and attracted extensive research attention. However, recent studies have shown that image caption models are vulnerable to some security threats such as backdoor attacks. Existing backdoor attacks against image captioning typically pair a trigger either with a predefined sentence or a single word as the targeted output, yet they are unrelated to the image content, making them easily noticeable as anomalies by humans. In this paper, we present a novel method to craft targeted backdoor attacks against image caption models, which are designed to be stealthier than prior attacks. Specifically, our method first learns a special trigger by leveraging universal perturbation techniques for object detection, then places the learned trigger in the center of some specific source object and modifies the corresponding object name in the output caption to a predefined target name. During the prediction phase, the caption produced by the backdoored model for input images with the trigger can accurately convey the semantic information of the rest of the whole image, while incorrectly recognizing the source object as the predefined target. Extensive experiments demonstrate that our approach can achieve a high attack success rate while having a negligible impact on model clean performance. In addition, we show our method is stealthy in that the produced backdoor samples are indistinguishable from clean samples in both image and text domains, which can successfully bypass existing backdoor defenses, highlighting the need for better defensive mechanisms against such stealthy backdoor attacks.
- [17] arXiv:2406.05892 [pdf, ps, html, other]
-
Title: Security Vulnerability Detection with Multitask Self-Instructed Fine-Tuning of Large Language ModelsSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG); Software Engineering (cs.SE)
Software security vulnerabilities allow attackers to perform malicious activities to disrupt software operations. Recent Transformer-based language models have significantly advanced vulnerability detection, surpassing the capabilities of static analysis based deep learning models. However, language models trained solely on code tokens do not capture either the explanation of vulnerability type or the data flow structure information of code, both of which are crucial for vulnerability detection. We propose a novel technique that integrates a multitask sequence-to-sequence LLM with pro-gram control flow graphs encoded as a graph neural network to achieve sequence-to-classification vulnerability detection. We introduce MSIVD, multitask self-instructed fine-tuning for vulnerability detection, inspired by chain-of-thought prompting and LLM self-instruction. Our experiments demonstrate that MSIVD achieves superior performance, outperforming the highest LLM-based vulnerability detector baseline (LineVul), with a F1 score of 0.92 on the BigVul dataset, and 0.48 on the PreciseBugs dataset. By training LLMs and GNNs simultaneously using a combination of code and explanatory metrics of a vulnerable program, MSIVD represents a promising direction for advancing LLM-based vulnerability detection that generalizes to unseen data. Based on our findings, we further discuss the necessity for new labelled security vulnerability datasets, as recent LLMs have seen or memorized prior datasets' held-out evaluation data.
- [18] arXiv:2406.05933 [pdf, ps, html, other]
-
Title: A Relevance Model for Threat-Centric Ranking of Cybersecurity VulnerabilitiesComments: 24 pages, 8 figures, 14 tablesSubjects: Cryptography and Security (cs.CR)
The relentless process of tracking and remediating vulnerabilities is a top concern for cybersecurity professionals. The key challenge is trying to identify a remediation scheme specific to in-house, organizational objectives. Without a strategy, the result is a patchwork of fixes applied to a tide of vulnerabilities, any one of which could be the point of failure in an otherwise formidable defense. Given that few vulnerabilities are a focus of real-world attacks, a practical remediation strategy is to identify vulnerabilities likely to be exploited and focus efforts towards remediating those vulnerabilities first. The goal of this research is to demonstrate that aggregating and synthesizing readily accessible, public data sources to provide personalized, automated recommendations for organizations to prioritize their vulnerability management strategy will offer significant improvements over using the Common Vulnerability Scoring System (CVSS). We provide a framework for vulnerability management specifically focused on mitigating threats using adversary criteria derived from MITRE ATT&CK. We test our approach by identifying vulnerabilities in software associated with six universities and four government facilities. Ranking policy performance is measured using the Normalized Discounted Cumulative Gain (nDCG). Our results show an average 71.5% - 91.3% improvement towards the identification of vulnerabilities likely to be targeted and exploited by cyber threat actors. The return on investment (ROI) of patching using our policies results in a savings of 23.3% - 25.5% in annualized costs. Our results demonstrate the efficacy of creating knowledge graphs to link large data sets to facilitate semantic queries and create data-driven, flexible ranking policies.
- [19] arXiv:2406.05941 [pdf, ps, html, other]
-
Title: Jailbreaking Quantum ComputersComments: 16 pages, 9 figuresSubjects: Cryptography and Security (cs.CR); Quantum Physics (quant-ph)
This work presented the first thorough exploration of the attacks on the interface between gate-level and pulse-level quantum circuits and pulse-level quantum circuits themselves. Typically, quantum circuits and programs that execute on quantum computers, are defined using gate-level primitives. However, to improve the expressivity of quantum circuits and to allow better optimization, pulse-level circuits are now often used. The attacks presented in this work leverage the inconsistency between the gate-level description of the custom gate, and the actual, low-level pulse implementation of this gate. By manipulating the custom gate specification, this work proposes numerous attacks: qubit plunder, qubit block, qubit reorder, timing mismatch, frequency mismatch, phase mismatch, and waveform mismatch. This work demonstrates these attacks on the real quantum computer and simulator, and shows that most current software development kits are vulnerable to these new types of attacks. In the end, this work proposes a defense framework. The exploration of security and privacy issues of the rising pulse-level quantum circuits provides insight into the future development of secure quantum software development kits and quantum computer systems.
- [20] arXiv:2406.05942 [pdf, ps, html, other]
-
Title: SETC: A Vulnerability Telemetry Collection FrameworkComments: 7 pages, 2 figuresSubjects: Cryptography and Security (cs.CR)
As emerging software vulnerabilities continuously threaten enterprises and Internet services, there is a critical need for improved security research capabilities. This paper introduces the Security Exploit Telemetry Collection (SETC) framework - an automated framework to generate reproducible vulnerability exploit data at scale for robust defensive security research. SETC deploys configurable environments to execute and record rich telemetry of vulnerability exploits within isolated containers. Exploits, vulnerable services, monitoring tools, and logging pipelines are defined via modular JSON configurations and deployed on demand. Compared to current manual processes, SETC enables automated, customizable, and repeatable vulnerability testing to produce diverse security telemetry. This research enables scalable exploit data generation to drive innovations in threat modeling, detection methods, analysis techniques, and remediation strategies. The capabilities of the framework are demonstrated through an example scenario. By addressing key barriers in security data generation, SETC represents a valuable platform to support impactful vulnerability and defensive security research.
- [21] arXiv:2406.05946 [pdf, ps, html, other]
-
Title: Safety Alignment Should Be Made More Than Just a Few Tokens DeepXiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, Peter HendersonSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
The safety alignment of current Large Language Models (LLMs) is vulnerable. Relatively simple attacks, or even benign fine-tuning, can jailbreak aligned models. We argue that many of these vulnerabilities are related to a shared underlying issue: safety alignment can take shortcuts, wherein the alignment adapts a model's generative distribution primarily over only its very first few output tokens. We refer to this issue as shallow safety alignment. In this paper, we present case studies to explain why shallow safety alignment can exist and provide evidence that current aligned LLMs are subject to this issue. We also show how these findings help explain multiple recently discovered vulnerabilities in LLMs, including the susceptibility to adversarial suffix attacks, prefilling attacks, decoding parameter attacks, and fine-tuning attacks. Importantly, we discuss how this consolidated notion of shallow safety alignment sheds light on promising research directions for mitigating these vulnerabilities. For instance, we show that deepening the safety alignment beyond just the first few tokens can often meaningfully improve robustness against some common exploits. Finally, we design a regularized finetuning objective that makes the safety alignment more persistent against fine-tuning attacks by constraining updates on initial tokens. Overall, we advocate that future safety alignment should be made more than just a few tokens deep.
- [22] arXiv:2406.05948 [pdf, ps, html, other]
-
Title: Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language ModelsSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
Backdoor attacks present significant threats to Large Language Models (LLMs), particularly with the rise of third-party services that offer API integration and prompt engineering. Untrustworthy third parties can plant backdoors into LLMs and pose risks to users by embedding malicious instructions into user queries. The backdoor-compromised LLM will generate malicious output when and input is embedded with a specific trigger predetermined by an attacker. Traditional defense strategies, which primarily involve model parameter fine-tuning and gradient calculation, are inadequate for LLMs due to their extensive computational and clean data requirements. In this paper, we propose a novel solution, Chain-of-Scrutiny (CoS), to address these challenges. Backdoor attacks fundamentally create a shortcut from the trigger to the target output, thus lack reasoning support. Accordingly, CoS guides the LLMs to generate detailed reasoning steps for the input, then scrutinizes the reasoning process to ensure consistency with the final answer. Any inconsistency may indicate an attack. CoS only requires black-box access to LLM, offering a practical defense, particularly for API-accessible LLMs. It is user-friendly, enabling users to conduct the defense themselves. Driven by natural language, the entire defense process is transparent to users. We validate the effectiveness of CoS through extensive experiments across various tasks and LLMs. Additionally, experiments results shows CoS proves more beneficial for more powerful LLMs.
- [23] arXiv:2406.06034 [pdf, ps, html, other]
-
Title: Shesha: Multi-head Microarchitectural Leakage Discovery in new-generation Intel ProcessorsComments: USENIX Security Symposium, 2024Subjects: Cryptography and Security (cs.CR)
Transient execution attacks have been one of the widely explored microarchitectural side channels since the discovery of Spectre and Meltdown. However, much of the research has been driven by manual discovery of new transient paths through well-known speculative events. Although a few attempts exist in literature on automating transient leakage discovery, such tools focus on finding variants of known transient attacks and explore a small subset of instruction set. Further, they take a random fuzzing approach that does not scale as the complexity of search space increases. In this work, we identify that the search space of bad speculation is disjointedly fragmented into equivalence classes, and then use this observation to develop a framework named Shesha, inspired by Particle Swarm Optimization, which exhibits faster convergence rates than state-of-the-art fuzzing techniques for automatic discovery of transient execution attacks. We then use Shesha to explore the vast search space of extensions to the x86 Instruction Set Architecture (ISEs), thereby focusing on previously unexplored avenues of bad speculation. As such, we report five previously unreported transient execution paths in Instruction Set Extensions (ISEs) on new generation of Intel processors. We then perform extensive reverse engineering of each of the transient execution paths and provide root-cause analysis. Using the discovered transient execution paths, we develop attack building blocks to exhibit exploitable transient windows. Finally, we demonstrate data leakage from Fused Multiply-Add instructions through SIMD buffer and extract victim data from various cryptographic implementations.
- [24] arXiv:2406.06099 [pdf, ps, html, other]
-
Title: Sequential Binary Classification for Intrusion Detection in Software Defined NetworksSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI)
Software-Defined Networks (SDN) are the standard architecture for network deployment. Intrusion Detection Systems (IDS) are a pivotal part of this technology as networks become more vulnerable to new and sophisticated attacks. Machine Learning (ML)-based IDS are increasingly seen as the most effective approach to handle this issue. However, IDS datasets suffer from high class imbalance, which impacts the performance of standard ML models. We propose Sequential Binary Classification (SBC) - an algorithm for multi-class classification to address this issue. SBC is a hierarchical cascade of base classifiers, each of which can be modelled on any general binary classifier. Extensive experiments are reported on benchmark datasets that evaluate the performance of SBC under different scenarios.
- [25] arXiv:2406.06164 [pdf, ps, html, other]
-
Title: Time to Separate from StackOverflow and Match with ChatGPT for EncryptionComments: Accepted in "Journal of Systems and Software", June 10, 2024Subjects: Cryptography and Security (cs.CR)
Cryptography is known as a challenging topic for developers. We studied StackOverflow posts to identify the problems that developers encounter when using Java Cryptography Architecture (JCA) for symmetric encryption. We investigated security risks that are disseminated in these posts, and we examined whether ChatGPT helps avoid cryptography issues. We found that developers frequently struggle with key and IV generations, as well as padding. Security is a top concern among developers, but security issues are pervasive in code snippets. ChatGPT can effectively aid developers when they engage with it properly. Nevertheless, it does not substitute human expertise, and developers should remain alert.
- [26] arXiv:2406.06186 [pdf, ps, html, other]
-
Title: A Survey on Machine Unlearning: Techniques and New Emerged Privacy RisksSubjects: Cryptography and Security (cs.CR)
The explosive growth of machine learning has made it a critical infrastructure in the era of artificial intelligence. The extensive use of data poses a significant threat to individual privacy. Various countries have implemented corresponding laws, such as GDPR, to protect individuals' data privacy and the right to be forgotten. This has made machine unlearning a research hotspot in the field of privacy protection in recent years, with the aim of efficiently removing the contribution and impact of individual data from trained models. The research in academia on machine unlearning has continuously enriched its theoretical foundation, and many methods have been proposed, targeting different data removal requests in various application scenarios. However, recently researchers have found potential privacy leakages of various of machine unlearning approaches, making the privacy preservation on machine unlearning area a critical topic. This paper provides an overview and analysis of the existing research on machine unlearning, aiming to present the current vulnerabilities of machine unlearning approaches. We analyze privacy risks in various aspects, including definitions, implementation methods, and real-world applications. Compared to existing reviews, we analyze the new challenges posed by the latest malicious attack techniques on machine unlearning from the perspective of privacy threats. We hope that this survey can provide an initial but comprehensive discussion on this new emerging area.
- [27] arXiv:2406.06225 [pdf, ps, html, other]
-
Title: Siren -- Advancing Cybersecurity through Deception and Adaptive AnalysisComments: 7 pages, 6 figuresSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Siren represents a pioneering research effort aimed at fortifying cybersecurity through strategic integration of deception, machine learning, and proactive threat analysis. Drawing inspiration from mythical sirens, this project employs sophisticated methods to lure potential threats into controlled environments. The system features a dynamic machine learning model for real-time analysis and classification, ensuring continuous adaptability to emerging cyber threats. The architectural framework includes a link monitoring proxy, a purpose-built machine learning model for dynamic link analysis, and a honeypot enriched with simulated user interactions to intensify threat engagement. Data protection within the honeypot is fortified with probabilistic encryption. Additionally, the incorporation of simulated user activity extends the system's capacity to capture and learn from potential attackers even after user disengagement. Siren introduces a paradigm shift in cybersecurity, transforming traditional defense mechanisms into proactive systems that actively engage and learn from potential adversaries. The research strives to enhance user protection while yielding valuable insights for ongoing refinement in response to the evolving landscape of cybersecurity threats.
- [28] arXiv:2406.06261 [pdf, ps, html, other]
-
Title: What All the PHUZZ Is About: A Coverage-guided Fuzzer for Finding Vulnerabilities in PHP Web ApplicationsComments: Preprint; Final version to be published in ASIA CCS '24: Proceedings of the 2024 ACM Asia Conference on Computer and Communications SecuritySubjects: Cryptography and Security (cs.CR)
Coverage-guided fuzz testing has received significant attention from the research community, with a strong focus on binary applications, greatly disregarding other targets, such as web applications. The importance of the World Wide Web in everyone's life cannot be overstated, and to this day, many web applications are developed in PHP. In this work, we address the challenges of applying coverage-guided fuzzing to PHP web applications and introduce PHUZZ, a modular fuzzing framework for PHP web applications. PHUZZ uses novel approaches to detect more client-side and server-side vulnerability classes than state-of-the-art related work, including SQL injections, remote command injections, insecure deserialization, path traversal, external entity injection, cross-site scripting, and open redirection. We evaluate PHUZZ on a diverse set of artificial and real-world web applications with known and unknown vulnerabilities, and compare it against a variety of state-of-the-art fuzzers. In order to show PHUZZ' effectiveness, we fuzz over 1,000 API endpoints of the 115 most popular WordPress plugins, resulting in over 20 security issues and 2 new CVE-IDs. Finally, we make the framework publicly available to motivate and encourage further research on web application fuzz testing.
- [29] arXiv:2406.06302 [pdf, ps, html, other]
-
Title: Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak AttacksSubjects: Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
The recent release of GPT-4o has garnered widespread attention due to its powerful general capabilities. While its impressive performance is widely acknowledged, its safety aspects have not been sufficiently explored. Given the potential societal impact of risky content generated by advanced generative AI such as GPT-4o, it is crucial to rigorously evaluate its safety. In response to this question, this paper for the first time conducts a rigorous evaluation of GPT-4o against jailbreak attacks. Specifically, this paper adopts a series of multi-modal and uni-modal jailbreak attacks on 4 commonly used benchmarks encompassing three modalities (\ie, text, speech, and image), which involves the optimization of over 4,000 initial text queries and the analysis and statistical evaluation of nearly 8,000+ response on GPT-4o. Our extensive experiments reveal several novel observations: (1) In contrast to the previous version (such as GPT-4V), GPT-4o has enhanced safety in the context of text modality jailbreak; (2) The newly introduced audio modality opens up new attack vectors for jailbreak attacks on GPT-4o; (3) Existing black-box multimodal jailbreak attack methods are largely ineffective against GPT-4o and GPT-4V. These findings provide critical insights into the safety implications of GPT-4o and underscore the need for robust alignment guardrails in large models. Our code is available at \url{this https URL}.
- [30] arXiv:2406.06429 [pdf, ps, html, other]
-
Title: A Note on Vectorial Boolean Functions as EmbeddingsComments: 12 pagesSubjects: Cryptography and Security (cs.CR)
Let $F$ be a vectorial Boolean function from $\mathbb{F}^n$ to $\mathbb{F}^m$, where $m \geq n$. We define $F$ as an embedding if $F$ is injective. In this paper, we examine the component functions of $F$, focusing on constant and balanced components. Our findings reveal that at most $2^m - 2^{m-n}$ components of $F$ can be balanced, and this maximum is achieved precisely when $F$ is an embedding, with the remaining $2^{m-n}$ components being constants. Additionally, for quadratic embeddings, we demonstrate that there are always at least $2^n - 1$ balanced components when $n$ is even, and $2^{m-1} + 2^{n-1} - 1$ balanced components when $n$ is odd.
- [31] arXiv:2406.06473 [pdf, ps, html, other]
-
Title: DiffAudit: Auditing Privacy Practices of Online Services for Children and AdolescentsSubjects: Cryptography and Security (cs.CR)
Children's and adolescents' online data privacy are regulated by laws such as the Children's Online Privacy Protection Act (COPPA) and the California Consumer Privacy Act (CCPA). Online services that are directed towards general audiences (i.e., including children, adolescents, and adults) must comply with these laws. In this paper, first, we present DiffAudit, a platform-agnostic privacy auditing methodology for general audience services. DiffAudit performs differential analysis of network traffic data flows to compare data processing practices (i) between child, adolescent, and adult users and (ii) before and after consent is given and user age is disclosed. We also present a data type classification method that utilizes GPT-4 and our data type ontology based on COPPA and CCPA, allowing us to identify considerably more data types than prior work. Second, we apply DiffAudit to a set of popular general audience mobile and web services and observe a rich set of behaviors extracted from over 440K outgoing requests, containing 3,968 unique data types we extracted and classified. We reveal problematic data processing practices prior to consent and age disclosure, lack of differentiation between age-specific data flows, inconsistent privacy policy disclosures, and sharing of linkable data with third parties, including advertising and tracking services.
- [32] arXiv:2406.06477 [pdf, ps, other]
-
Title: OmniLytics+: A Secure, Efficient, and Affordable Blockchain Data Market for Machine Learning through Off-Chain ProcessingSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
The rapid development of large machine learning (ML) models requires a massive amount of training data, resulting in booming demands of data sharing and trading through data markets. Traditional centralized data markets suffer from low level of security, and emerging decentralized platforms are faced with efficiency and privacy challenges. In this paper, we propose OmniLytics+, the first decentralized data market, built upon blockchain and smart contract technologies, to simultaneously achieve 1) data (resp., model) privacy for the data (resp. model) owner; 2) robustness against malicious data owners; 3) efficient data validation and aggregation. Specifically, adopting the zero-knowledge (ZK) rollup paradigm, OmniLytics+ proposes to secret share encrypted local gradients, computed from the encrypted global model, with a set of untrusted off-chain servers, who collaboratively generate a ZK proof on the validity of the gradient. In this way, the storage and processing overheads are securely offloaded from blockchain verifiers, significantly improving the privacy, efficiency, and affordability over existing rollup solutions. We implement the proposed OmniLytics+ data market as an Ethereum smart contract [41]. Extensive experiments demonstrate the effectiveness of OmniLytics+ in training large ML models in presence of malicious data owner, and the substantial advantages of OmniLytics+ in gas cost and execution time over baselines.
New submissions for Tuesday, 11 June 2024 (showing 32 of 32 entries )
- [33] arXiv:2406.05257 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Efficient Differentially Private Fine-Tuning of Diffusion ModelsSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
The recent developments of Diffusion Models (DMs) enable generation of astonishingly high-quality synthetic samples. Recent work showed that the synthetic samples generated by the diffusion model, which is pre-trained on public data and fully fine-tuned with differential privacy on private data, can train a downstream classifier, while achieving a good privacy-utility tradeoff. However, fully fine-tuning such large diffusion models with DP-SGD can be very resource-demanding in terms of memory usage and computation. In this work, we investigate Parameter-Efficient Fine-Tuning (PEFT) of diffusion models using Low-Dimensional Adaptation (LoDA) with Differential Privacy. We evaluate the proposed method with the MNIST and CIFAR-10 datasets and demonstrate that such efficient fine-tuning can also generate useful synthetic samples for training downstream classifiers, with guaranteed privacy protection of fine-tuning data. Our source code will be made available on GitHub.
- [34] arXiv:2406.05264 (cross-list from stat.AP) [pdf, ps, html, other]
-
Title: "Minus-One" Data Prediction Generates Synthetic Census Data with Good Crosstabulation FidelityComments: 35 pages, 17 figures, 6 tablesSubjects: Applications (stat.AP); Cryptography and Security (cs.CR); Computers and Society (cs.CY); Methodology (stat.ME)
We propose to capture relevant statistical associations in a dataset of categorical survey responses by a method, here termed MODP, that "learns" a probabilistic prediction function L. Specifically, L predicts each question's response based on the same respondent's answers to all the other questions. Draws from the resulting probability distribution become synthetic responses. Applying this methodology to the PUMS subset of Census ACS data, and with a learned L akin to multiple parallel logistic regression, we generate synthetic responses whose crosstabulations (two-point conditionals) are found to have a median accuracy of ~5% across all crosstabulation cells, with cell counts ranging over four orders of magnitude. We investigate and attempt to quantify the degree to which the privacy of the original data is protected.
- [35] arXiv:2406.05491 (cross-list from cs.CV) [pdf, ps, html, other]
-
Title: One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training ModelsSubjects: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
Vision-Language Pre-training (VLP) models trained on large-scale image-text pairs have demonstrated unprecedented capability in many practical applications. However, previous studies have revealed that VLP models are vulnerable to adversarial samples crafted by a malicious adversary. While existing attacks have achieved great success in improving attack effect and transferability, they all focus on instance-specific attacks that generate perturbations for each input sample. In this paper, we show that VLP models can be vulnerable to a new class of universal adversarial perturbation (UAP) for all input samples. Although initially transplanting existing UAP algorithms to perform attacks showed effectiveness in attacking discriminative models, the results were unsatisfactory when applied to VLP models. To this end, we revisit the multimodal alignments in VLP model training and propose the Contrastive-training Perturbation Generator with Cross-modal conditions (C-PGC). Specifically, we first design a generator that incorporates cross-modal information as conditioning input to guide the training. To further exploit cross-modal interactions, we propose to formulate the training objective as a multimodal contrastive learning paradigm based on our constructed positive and negative image-text pairs. By training the conditional generator with the designed loss, we successfully force the adversarial samples to move away from its original area in the VLP model's feature space, and thus essentially enhance the attacks. Extensive experiments show that our method achieves remarkable attack performance across various VLP models and Vision-and-Language (V+L) tasks. Moreover, C-PGC exhibits outstanding black-box transferability and achieves impressive results in fooling prevalent large VLP models including LLaVA and Qwen-VL.
- [36] arXiv:2406.05535 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Perturbation Towards Easy Samples Improves Targeted Adversarial TransferabilityJournal-ref: Advances in Neural Information Processing Systems 36, 2023Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
The transferability of adversarial perturbations provides an effective shortcut for black-box attacks. Targeted perturbations have greater practicality but are more difficult to transfer between models. In this paper, we experimentally and theoretically demonstrated that neural networks trained on the same dataset have more consistent performance in High-Sample-Density-Regions (HSDR) of each class instead of low sample density regions. Therefore, in the target setting, adding perturbations towards HSDR of the target class is more effective in improving transferability. However, density estimation is challenging in high-dimensional scenarios. Further theoretical and experimental verification demonstrates that easy samples with low loss are more likely to be located in HSDR. Perturbations towards such easy samples in the target class can avoid density estimation for HSDR location. Based on the above facts, we verified that adding perturbations to easy samples in the target class improves targeted adversarial transferability of existing attack methods. A generative targeted attack strategy named Easy Sample Matching Attack (ESMA) is proposed, which has a higher success rate for targeted attacks and outperforms the SOTA generative method. Moreover, ESMA requires only 5% of the storage space and much less computation time comparing to the current SOTA, as ESMA attacks all classes with only one model instead of seperate models for each class. Our code is available at this https URL.
- [37] arXiv:2406.05545 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Privacy-Preserving Optimal Parameter Selection for Collaborative ClusteringSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
This study investigates the optimal selection of parameters for collaborative clustering while ensuring data privacy. We focus on key clustering algorithms within a collaborative framework, where multiple data owners combine their data. A semi-trusted server assists in recommending the most suitable clustering algorithm and its parameters. Our findings indicate that the privacy parameter ($\epsilon$) minimally impacts the server's recommendations, but an increase in $\epsilon$ raises the risk of membership inference attacks, where sensitive information might be inferred. To mitigate these risks, we implement differential privacy techniques, particularly the Randomized Response mechanism, to add noise and protect data privacy. Our approach demonstrates that high-quality clustering can be achieved while maintaining data confidentiality, as evidenced by metrics such as the Adjusted Rand Index and Silhouette Score. This study contributes to privacy-aware data sharing, optimal algorithm and parameter selection, and effective communication between data owners and the server.
- [38] arXiv:2406.05568 (cross-list from cs.DC) [pdf, ps, html, other]
-
Title: SAMM: Sharded Automated Market MakersSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Cryptography and Security (cs.CR)
\emph{Automated Market Makers} (\emph{AMMs}) are a cornerstone of decentralized finance (DeFi) blockchain-based platforms.
They are smart contracts, enabling the direct exchange of virtual tokens by maintaining \emph{liquidity pools}.
Traders exchange tokens with the contract, paying a fee; liquidity comes from \emph{liquidity providers}, paid by those fees.
But despite growing demand, the performance of AMMs is limited.
State-of-the-art blockchain platforms allow for parallel execution of transactions.
However, we show that AMMs do not enjoy these gains, since their operations are not commutative so transactions using them must be serialized.
We present \emph{SAMM}, an AMM comprising multiple independent \emph{shards}.
All shards are smart contracts operating in the same chain, but they allow for parallel execution as each is independent.
The challenge is that trading in a standard AMM is cheaper if its liquidity pool is larger.
Therefore, we show that simply using multiple smaller AMMs results in traders splitting each trade among all AMMs, which worsens performance.
SAMM addresses this issue with a novel design of the trading fees.
Traders are incentivized to use only a single smallest shard.
We show that all Subgame-Perfect Nash Equilibria (SPNE) fit the desired behavior: Liquidity providers balance the liquidity among all pools, so the system converges to the state where trades are evenly distributed.
Evaluation in the Sui blockchain shows that SAMM's throughput is over fivefold that of traditional AMMs, approaching the system's limit.
SAMM is a directly deployable open-source smart contract, allowing trading at scale for individuals and DeFi applications. - [39] arXiv:2406.05644 (cross-list from cs.CL) [pdf, ps, other]
-
Title: How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden StatesComments: 27 pagesSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Computers and Society (cs.CY)
Large language models (LLMs) rely on safety alignment to avoid responding to malicious user inputs. Unfortunately, jailbreak can circumvent safety guardrails, resulting in LLMs generating harmful content and raising concerns about LLM safety. Due to language models with intensive parameters often regarded as black boxes, the mechanisms of alignment and jailbreak are challenging to elucidate. In this paper, we employ weak classifiers to explain LLM safety through the intermediate hidden states. We first confirm that LLMs learn ethical concepts during pre-training rather than alignment and can identify malicious and normal inputs in the early layers. Alignment actually associates the early concepts with emotion guesses in the middle layers and then refines them to the specific reject tokens for safe generations. Jailbreak disturbs the transformation of early unethical classification into negative emotions. We conduct experiments on models from 7B to 70B across various model families to prove our conclusion. Overall, our paper indicates the intrinsical mechanism of LLM safety and how jailbreaks circumvent safety guardrails, offering a new perspective on LLM safety and reducing concerns.
- [40] arXiv:2406.05660 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Injecting Undetectable Backdoors in Deep Learning and Language ModelsAlkis Kalavasis, Amin Karbasi, Argyris Oikonomou, Katerina Sotiraki, Grigoris Velegkas, Manolis ZampetakisSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)
As ML models become increasingly complex and integral to high-stakes domains such as finance and healthcare, they also become more susceptible to sophisticated adversarial attacks. We investigate the threat posed by undetectable backdoors in models developed by insidious external expert firms. When such backdoors exist, they allow the designer of the model to sell information to the users on how to carefully perturb the least significant bits of their input to change the classification outcome to a favorable one. We develop a general strategy to plant a backdoor to neural networks while ensuring that even if the model's weights and architecture are accessible, the existence of the backdoor is still undetectable. To achieve this, we utilize techniques from cryptography such as cryptographic signatures and indistinguishability obfuscation. We further introduce the notion of undetectable backdoors to language models and extend our neural network backdoor attacks to such models based on the existence of steganographic functions.
- [41] arXiv:2406.05670 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Certified Robustness to Data Poisoning in Gradient-Based TrainingComments: 15 pages, 5 figuresSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
Modern machine learning pipelines leverage large amounts of public data, making it infeasible to guarantee data quality and leaving models open to poisoning and backdoor attacks. However, provably bounding model behavior under such attacks remains an open problem. In this work, we address this challenge and develop the first framework providing provable guarantees on the behavior of models trained with potentially manipulated data. In particular, our framework certifies robustness against untargeted and targeted poisoning as well as backdoor attacks for both input and label manipulations. Our method leverages convex relaxations to over-approximate the set of all possible parameter updates for a given poisoning threat model, allowing us to bound the set of all reachable parameters for any gradient-based learning algorithm. Given this set of parameters, we provide bounds on worst-case behavior, including model performance and backdoor success rate. We demonstrate our approach on multiple real-world datasets from applications including energy consumption, medical imaging, and autonomous driving.
- [42] arXiv:2406.05726 (cross-list from cs.CV) [pdf, ps, html, other]
-
Title: Region of Interest Loss for Anonymizing Learned Image CompressionComments: Accepted to IEEE CASE 2024Subjects: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR); Machine Learning (cs.LG); Image and Video Processing (eess.IV)
The use of AI in public spaces continually raises concerns about privacy and the protection of sensitive data. An example is the deployment of detection and recognition methods on humans, where images are provided by surveillance cameras. This results in the acquisition of great amounts of sensitive data, since the capture and transmission of images taken by such cameras happens unaltered, for them to be received by a server on the network. However, many applications do not explicitly require the identity of a given person in a scene; An anonymized representation containing information of the person's position while preserving the context of them in the scene suffices. We show how using a customized loss function on region of interests (ROI) can achieve sufficient anonymization such that human faces become unrecognizable while persons are kept detectable, by training an end-to-end optimized autoencoder for learned image compression that utilizes the flexibility of the learned analysis and reconstruction transforms for the task of mutating parts of the compression result. This approach enables compression and anonymization in one step on the capture device, instead of transmitting sensitive, nonanonymized data over the network. Additionally, we evaluate how this anonymization impacts the average precision of pre-trained foundation models on detecting faces (MTCNN) and humans (YOLOv8) in comparison to non-ANN based methods, while considering compression rate and latency.
- [43] arXiv:2406.05800 (cross-list from cs.CV) [pdf, ps, html, other]
-
Title: SlowPerception: Physical-World Latency Attack against Visual Perception in Autonomous DrivingSubjects: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR)
Autonomous Driving (AD) systems critically depend on visual perception for real-time object detection and multiple object tracking (MOT) to ensure safe driving. However, high latency in these visual perception components can lead to significant safety risks, such as vehicle collisions. While previous research has extensively explored latency attacks within the digital realm, translating these methods effectively to the physical world presents challenges. For instance, existing attacks rely on perturbations that are unrealistic or impractical for AD, such as adversarial perturbations affecting areas like the sky, or requiring large patches that obscure most of a camera's view, thus making them impossible to be conducted effectively in the real world.
In this paper, we introduce SlowPerception, the first physical-world latency attack against AD perception, via generating projector-based universal perturbations. SlowPerception strategically creates numerous phantom objects on various surfaces in the environment, significantly increasing the computational load of Non-Maximum Suppression (NMS) and MOT, thereby inducing substantial latency. Our SlowPerception achieves second-level latency in physical-world settings, with an average latency of 2.5 seconds across different AD perception systems, scenarios, and hardware configurations. This performance significantly outperforms existing state-of-the-art latency attacks. Additionally, we conduct AD system-level impact assessments, such as vehicle collisions, using industry-grade AD systems with production-grade AD simulators with a 97% average rate. We hope that our analyses can inspire further research in this critical domain, enhancing the robustness of AD systems against emerging vulnerabilities. - [44] arXiv:2406.05826 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: PSBD: Prediction Shift Uncertainty Unlocks Backdoor DetectionSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR)
Deep neural networks are susceptible to backdoor attacks, where adversaries manipulate model predictions by inserting malicious samples into the training data. Currently, there is still a lack of direct filtering methods for identifying suspicious training data to unveil potential backdoor samples. In this paper, we propose a novel method, Prediction Shift Backdoor Detection (PSBD), leveraging an uncertainty-based approach requiring minimal unlabeled clean validation data. PSBD is motivated by an intriguing Prediction Shift (PS) phenomenon, where poisoned models' predictions on clean data often shift away from true labels towards certain other labels with dropout applied during inference, while backdoor samples exhibit less PS. We hypothesize PS results from neuron bias effect, making neurons favor features of certain classes. PSBD identifies backdoor training samples by computing the Prediction Shift Uncertainty (PSU), the variance in probability values when dropout layers are toggled on and off during model inference. Extensive experiments have been conducted to verify the effectiveness and efficiency of PSBD, which achieves state-of-the-art results among mainstream detection methods.
- [45] arXiv:2406.05858 (cross-list from cs.DC) [pdf, ps, html, other]
-
Title: Comments on "Federated Learning with Differential Privacy: Algorithms and Performance Analysis"Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Cryptography and Security (cs.CR); Performance (cs.PF)
In the paper by Wei et al. ("Federated Learning with Differential Privacy: Algorithms and Performance Analysis"), the convergence performance of the proposed differential privacy algorithm in federated learning (FL), known as Noising before Model Aggregation FL (NbAFL), was studied. However, the presented convergence upper bound of NbAFL (Theorem 2) is incorrect. This comment aims to present the correct form of the convergence upper bound for NbAFL.
- [46] arXiv:2406.05904 (cross-list from cs.DC) [pdf, ps, html, other]
-
Title: Aegis: A Decentralized Expansion BlockchainSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Cryptography and Security (cs.CR)
Blockchains implement monetary systems operated by committees of nodes. The robustness of established blockchains presents an opportunity to leverage their infrastructure for creating expansion chains. Expansion chains can provide additional functionality to the primary chain they leverage or implement separate functionalities, while benefiting from the primary chain's security and the stability of its tokens. Indeed, tools like Ethereum's EigenLayer enable nodes to stake (deposit collateral) on a primary chain to form a committee responsible for operating an expansion chain.
But here is the rub. Classical protocols assume correct, well-behaved nodes stay correct indefinitely. Yet in our case, the stake incentivizes correctness--it will be slashed (revoked) if its owner deviates. Once a node withdraws its stake, there is no basis to assume its correctness.
To address the new challenge, we present Aegis, an expansion chain based on primary-chain stake, assuming a bounded primary-chain write time. Aegis uses references from Aegis blocks to primary blocks to define committees, checkpoints on the primary chain to perpetuate decisions, and resets on the primary chain to establish a new committee if the previous one becomes obsolete. It ensures safety at all times and rapid progress when latency among Aegis nodes is low. - [47] arXiv:2406.05927 (cross-list from cs.CV) [pdf, ps, html, other]
-
Title: MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature SparsificationSubjects: Computer Vision and Pattern Recognition (cs.CV); Cryptography and Security (cs.CR); Machine Learning (cs.LG)
We present a simple yet effective method to improve the robustness of Convolutional Neural Networks (CNNs) against adversarial examples by post-processing an adversarially trained model. Our technique, MeanSparse, cascades the activation functions of a trained model with novel operators that sparsify mean-centered feature vectors. This is equivalent to reducing feature variations around the mean, and we show that such reduced variations merely affect the model's utility, yet they strongly attenuate the adversarial perturbations and decrease the attacker's success rate. Our experiments show that, when applied to the top models in the RobustBench leaderboard, it achieves a new robustness record of 72.08% (from 71.07%) and 59.64% (from 59.56%) on CIFAR-10 and ImageNet, respectively, in term of AutoAttack accuracy. Code is available at this https URL
- [48] arXiv:2406.05929 (cross-list from cs.CY) [pdf, ps, html, other]
-
Title: Cyber-sensorium: An Extension of the Cyber Public Health FrameworkSubjects: Computers and Society (cs.CY); Cryptography and Security (cs.CR)
In response to increasingly sophisticated cyberattacks, a health-based approach is being used to define and assess their impact. Two significant cybersecurity workshops have fostered this perspective, aiming to standardize the understanding of cyber harm. Experts at these workshops agreed on a public health-like framework to analyze cyber threats focusing on the perpetrators' intent, the means available to them, and the vulnerability of targets. We contribute to this dialogue with the cyber sensorium concept, drawing parallels between the digital network and a biological nervous system essential to human welfare. Cyberattacks on this system present serious global risks, underlining the need for its protection.
- [49] arXiv:2406.06153 (cross-list from cs.SE) [pdf, ps, html, other]
-
Title: Gameful Introduction to Cryptography for Dyslexic StudentsComments: 36th IEEE Conference on Software Engineering Education and Training (CSEE&T 2024)Subjects: Software Engineering (cs.SE); Cryptography and Security (cs.CR)
Cryptography has a pivotal role in securing our digital world. Nonetheless, it is a challenging topic to learn. In this paper, we show that despite its complex nature, dyslexia$-$a learning disorder that influences reading and writing skills$-$does not hinder one's ability to comprehend cryptography. In particular, we conducted a gameful workshop with 14 high-school dyslexic students and taught them fundamental encryption methods. The students engaged well, learned the techniques, and enjoyed the training. We conclude that with a proper approach, dyslexia cannot hinder learning a complex subject such as cryptography.
- [50] arXiv:2406.06207 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: Lurking in the shadows: Unveiling Stealthy Backdoor Attacks against Personalized Federated LearningXiaoting Lyu, Yufei Han, Wei Wang, Jingkai Liu, Yongsheng Zhu, Guangquan Xu, Jiqiang Liu, Xiangliang ZhangComments: Accepted by Usenix Security 2024Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR)
Federated Learning (FL) is a collaborative machine learning technique where multiple clients work together with a central server to train a global model without sharing their private data. However, the distribution shift across non-IID datasets of clients poses a challenge to this one-model-fits-all method hindering the ability of the global model to effectively adapt to each client's unique local data. To echo this challenge, personalized FL (PFL) is designed to allow each client to create personalized local models tailored to their private data. While extensive research has scrutinized backdoor risks in FL, it has remained underexplored in PFL applications. In this study, we delve deep into the vulnerabilities of PFL to backdoor attacks. Our analysis showcases a tale of two cities. On the one hand, the personalization process in PFL can dilute the backdoor poisoning effects injected into the personalized local models. Furthermore, PFL systems can also deploy both server-end and client-end defense mechanisms to strengthen the barrier against backdoor attacks. On the other hand, our study shows that PFL fortified with these defense methods may offer a false sense of security. We propose \textit{PFedBA}, a stealthy and effective backdoor attack strategy applicable to PFL systems. \textit{PFedBA} ingeniously aligns the backdoor learning task with the main learning task of PFL by optimizing the trigger generation process. Our comprehensive experiments demonstrate the effectiveness of \textit{PFedBA} in seamlessly embedding triggers into personalized local models. \textit{PFedBA} yields outstanding attack performance across 10 state-of-the-art PFL algorithms, defeating the existing 6 defense mechanisms. Our study sheds light on the subtle yet potent backdoor threats to PFL systems, urging the community to bolster defenses against emerging backdoor challenges.
- [51] arXiv:2406.06231 (cross-list from math.ST) [pdf, ps, html, other]
-
Title: Statistical Inference for Privatized Data with Unknown Sample SizeComments: 20 pages before references, 40 pages in total, 4 figures, 3 tablesSubjects: Statistics Theory (math.ST); Cryptography and Security (cs.CR); Computation (stat.CO)
We develop both theory and algorithms to analyze privatized data in the unbounded differential privacy(DP), where even the sample size is considered a sensitive quantity that requires privacy protection. We show that the distance between the sampling distributions under unbounded DP and bounded DP goes to zero as the sample size $n$ goes to infinity, provided that the noise used to privatize $n$ is at an appropriate rate; we also establish that ABC-type posterior distributions converge under similar assumptions. We further give asymptotic results in the regime where the privacy budget for $n$ goes to zero, establishing similarity of sampling distributions as well as showing that the MLE in the unbounded setting converges to the bounded-DP MLE. In order to facilitate valid, finite-sample Bayesian inference on privatized data in the unbounded DP setting, we propose a reversible jump MCMC algorithm which extends the data augmentation MCMC of Ju et al. (2022). We also propose a Monte Carlo EM algorithm to compute the MLE from privatized data in both bounded and unbounded DP. We apply our methodology to analyze a linear regression model as well as a 2019 American Time Use Survey Microdata File which we model using a Dirichlet distribution.
- [52] arXiv:2406.06252 (cross-list from eess.SP) [pdf, ps, html, other]
-
Title: Random Time-hopping Secure Ranging Strategy Against Distance-Reduction Attacks in UWBSubjects: Signal Processing (eess.SP); Cryptography and Security (cs.CR)
In order to mitigate the distance reduction attack in Ultra-Wide Band (UWB) ranging, this paper proposes a secure ranging scheme based on a random time-hopping mechanism without redundant signaling overhead. Additionally, a secure ranging strategy is designed for backward compatibility with existing standards such as IEEE 802.15.4a/z, combined with an attack detection scheme. The effectiveness and feasibility of the proposed strategy are demonstrated through both simulation and experimental results in the case of the Ghost Peak attack, as demonstrated by Patrick Leu et al. The random time-hopping mechanism is verified to be capable of reducing the success rate of distance reduction attacks to less than 0.01%, thereby significantly enhancing the security of UWB ranging.
- [53] arXiv:2406.06318 (cross-list from cs.DC) [pdf, ps, other]
-
Title: Should my Blockchain Learn to Drive? A Study of Hyperledger FabricSubjects: Distributed, Parallel, and Cluster Computing (cs.DC); Cryptography and Security (cs.CR)
Similar to other transaction processing frameworks, blockchain systems need to be dynamically reconfigured to adapt to varying workloads and changes in network conditions. However, achieving optimal reconfiguration is particularly challenging due to the complexity of the blockchain stack, which has diverse configurable parameters. This paper explores the concept of self-driving blockchains, which have the potential to predict workload changes and reconfigure themselves for optimal performance without human intervention. We compare and contrast our discussions with existing research on databases and highlight aspects unique to blockchains. We identify specific parameters and components in Hyperledger Fabric, a popular permissioned blockchain system, that are suitable for autonomous adaptation and offer potential solutions for the challenges involved. Further, we implement three demonstrative locally autonomous systems, each targeting a different layer of the blockchain stack, and conduct experiments to understand the feasibility of our findings. Our experiments indicate up to 11% improvement in success throughput and a 30% decrease in latency, making this a significant step towards implementing a fully autonomous blockchain system in the future.
- [54] arXiv:2406.06408 (cross-list from stat.ML) [pdf, ps, other]
-
Title: Differentially Private Best-Arm IdentificationComments: arXiv admin note: substantial text overlap with arXiv:2309.02202Subjects: Machine Learning (stat.ML); Cryptography and Security (cs.CR); Machine Learning (cs.LG); Statistics Theory (math.ST)
Best Arm Identification (BAI) problems are progressively used for data-sensitive applications, such as designing adaptive clinical trials, tuning hyper-parameters, and conducting user studies. Motivated by the data privacy concerns invoked by these applications, we study the problem of BAI with fixed confidence in both the local and central models, i.e. $\epsilon$-local and $\epsilon$-global Differential Privacy (DP). First, to quantify the cost of privacy, we derive lower bounds on the sample complexity of any $\delta$-correct BAI algorithm satisfying $\epsilon$-global DP or $\epsilon$-local DP. Our lower bounds suggest the existence of two privacy regimes. In the high-privacy regime, the hardness depends on a coupled effect of privacy and novel information-theoretic quantities involving the Total Variation. In the low-privacy regime, the lower bounds reduce to the non-private lower bounds. We propose $\epsilon$-local DP and $\epsilon$-global DP variants of a Top Two algorithm, namely CTB-TT and AdaP-TT*, respectively. For $\epsilon$-local DP, CTB-TT is asymptotically optimal by plugging in a private estimator of the means based on Randomised Response. For $\epsilon$-global DP, our private estimator of the mean runs in arm-dependent adaptive episodes and adds Laplace noise to ensure a good privacy-utility trade-off. By adapting the transportation costs, the expected sample complexity of AdaP-TT* reaches the asymptotic lower bound up to multiplicative constants.
- [55] arXiv:2406.06443 (cross-list from cs.LG) [pdf, ps, html, other]
-
Title: LLM Dataset Inference: Did you train on my dataset?Comments: Code is available at \href{this https URLSubjects: Machine Learning (cs.LG); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
The proliferation of large language models (LLMs) in the real world has come with a rise in copyright cases against companies for training their models on unlicensed data from the internet. Recent works have presented methods to identify if individual text sequences were members of the model's training data, known as membership inference attacks (MIAs). We demonstrate that the apparent success of these MIAs is confounded by selecting non-members (text sequences not used for training) belonging to a different distribution from the members (e.g., temporally shifted recent Wikipedia articles compared with ones used to train the model). This distribution shift makes membership inference appear successful. However, most MIA methods perform no better than random guessing when discriminating between members and non-members from the same distribution (e.g., in this case, the same period of time). Even when MIAs work, we find that different MIAs succeed at inferring membership of samples from different distributions. Instead, we propose a new dataset inference method to accurately identify the datasets used to train large language models. This paradigm sits realistically in the modern-day copyright landscape, where authors claim that an LLM is trained over multiple documents (such as a book) written by them, rather than one particular paragraph. While dataset inference shares many of the challenges of membership inference, we solve it by selectively combining the MIAs that provide positive signal for a given distribution, and aggregating them to perform a statistical test on a given dataset. Our approach successfully distinguishes the train and test sets of different subsets of the Pile with statistically significant p-values < 0.1, without any false positives.
- [56] arXiv:2406.06483 (cross-list from cs.NI) [pdf, ps, html, other]
-
Title: A Taxonomy and Comparative Analysis of IPv4 ID Selection Correctness, Security, and PerformanceJoshua J. Daymude, Antonio M. Espinoza, Sean Bergen, Benjamin Mixon-Baca, Jeffrey Knockel, Jedidiah R. CrandallComments: 31 pages, 10 figures, 1 table, 1 algorithmSubjects: Networking and Internet Architecture (cs.NI); Cryptography and Security (cs.CR)
The battle for a more secure Internet is waged on many fronts, including the most basic of networking protocols. Our focus is the IPv4 Identifier (IPID), an IPv4 header field as old as the Internet with an equally long history as an exploited side channel for scanning network properties, inferring off-path connections, and poisoning DNS caches. This article taxonomizes the 25-year history of IPID-based exploits and the corresponding changes to IPID selection methods. By mathematically analyzing these methods' correctness and security and empirically evaluating their performance, we reveal recommendations for best practice as well as shortcomings of current operating system implementations, emphasizing the value of systematic evaluations in network security.
Cross submissions for Tuesday, 11 June 2024 (showing 24 of 24 entries )
- [57] arXiv:2301.10733 (replaced) [pdf, ps, other]
-
Title: The Synchronic WebSubjects: Cryptography and Security (cs.CR)
The Synchronic Web is a distributed network for securing data provenance on the World Wide Web. By enabling clients around the world to freely commit digital information into a single shared view of history, it provides a foundational basis of truth on which to build decentralized and scalable trust across the Internet. Its core cryptographical capability allows mutually distrusting parties to create and verify statements of the following form: "I commit to this information--and only this information--at this moment in time." The backbone of the Synchronic Web infrastructure is a simple, small, and semantic-free blockchain that is accessible to any Internet-enabled entity. The infrastructure is maintained by a permissioned network of well-known servers, called notaries, and accessed by a permissionless group of clients, called ledgers. Through an evolving stack of flexible and composable semantic specifications, the parties cooperate to generate synchronic commitments over arbitrary data. When integrated with existing infrastructures, adapted to diverse domains, and scaled across the breadth of cyberspace, the Synchronic Web provides a ubiquitous mechanism to lock the world's data into unique points in discrete time and digital space.
- [58] arXiv:2304.14540 (replaced) [pdf, ps, html, other]
-
Title: Interactive Greybox Penetration Testing for Cloud Access Control using IAM Modeling and Deep Reinforcement LearningSubjects: Cryptography and Security (cs.CR); Software Engineering (cs.SE)
Identity and Access Management (IAM) is an access control service in cloud platforms. To securely manage cloud resources, customers need to configure IAM to specify the access control rules for their cloud organizations. However, incorrectly configured IAM can be exploited to cause a security attack such as privilege escalation (PE), leading to severe economic loss. To detect such PEs due to IAM misconfigurations, third-party cloud security services are commonly used. The state-of-the-art services apply whitebox penetration testing techniques, which require access to complete IAM configurations. However, the configurations can contain sensitive information. To prevent the disclosure of such information, customers need to manually anonymize the configuration.
In this paper, we propose a precise greybox penetration testing approach called TAC for third-party services to detect IAM PEs. To mitigate the dual challenges of labor-intensive anonymization and potentially sensitive information disclosures, TAC interacts with customers by selectively querying only the essential information needed. Our key insight is that only a small fraction of information in the IAM configuration is relevant to the IAM PE detection. We first propose IAM modeling, enabling TAC to detect a broad class of IAM PEs based on the partial information collected from queries. To improve the efficiency and applicability of TAC, we aim to minimize interactions with customers by applying Reinforcement Learning (RL) with Graph Neural Networks (GNNs), allowing TAC to learn to make as few queries as possible. Experimental results on both synthetic and real-world tasks show that, compared to state-of-the-art whitebox approaches, TAC detects IAM PEs with competitively low false negative rates, employing a limited number of queries. - [59] arXiv:2306.05923 (replaced) [pdf, ps, html, other]
-
Title: When Authentication Is Not Enough: On the Security of Behavioral-Based Driver Authentication SystemsSubjects: Cryptography and Security (cs.CR)
Many research papers have recently focused on behavioral-based driver authentication systems in vehicles. Pushed by Artificial Intelligence (AI) advancements, these works propose powerful models to identify drivers through their unique biometric behavior. However, these models have never been scrutinized from a security point of view, rather focusing on the performance of the AI algorithms. Several limitations and oversights make implementing the state-of-the-art impractical, such as their secure connection to the vehicle's network and the management of security alerts. Furthermore, due to the extensive use of AI, these systems may be vulnerable to adversarial attacks. However, there is currently no discussion on the feasibility and impact of such attacks in this scenario.
Driven by the significant gap between research and practical application, this paper seeks to connect these two domains. We propose the first security-aware system model for behavioral-based driver authentication. We develop two lightweight driver authentication systems based on Random Forest and Recurrent Neural Network architectures designed for our constrained environments. We formalize a realistic system and threat model reflecting a real-world vehicle's network for their implementation. When evaluated on real driving data, our models outclass the state-of-the-art with an accuracy of up to 0.999 in identification and authentication. Moreover, we are the first to propose attacks against these systems by developing two novel evasion attacks, SMARTCAN and GANCAN. We show how attackers can still exploit these systems with a perfect attack success rate (up to 1.000). Finally, we discuss requirements for deploying driver authentication systems securely. Through our contributions, we aid practitioners in safely adopting these systems, help reduce car thefts, and enhance driver security. - [60] arXiv:2310.02530 (replaced) [pdf, ps, html, other]
-
Title: CompVPD: Iteratively Identifying Vulnerability Patches Based on Human Validation Results with a Precise ContextTianyu Chen, Lin Li, Taotao Qian, Jingyi Liu, Wei Yang, Ding Li, Guangtai Liang, Qianxiang Wang, Tao XieSubjects: Cryptography and Security (cs.CR)
Applying security patches in open source software timely is critical for ensuring the security of downstream applications. However, it is challenging to apply these patches promptly because notifications of patches are often incomplete and delayed. To address this issue, existing approaches employ deep-learning (DL) models to identify additional vulnerability patches by determining whether a code commit addresses a vulnerability. Nonetheless, these approaches suffer from low accuracy due to the imprecise context provided for the patches. To provide precise context for patches, we propose a multi-granularity slicing algorithm and an adaptive-expanding algorithm to accurately identify code related to the patches. Additionally, the precise context enables to design an iterative identification framework, CompVPD, which utilizes the human validation results, and substantially improve the effectiveness. We empirically compare CompVPD with four state-of-the-art/practice (SOTA) approaches in identifying vulnerability patches. The results demonstrate that CompVPD improves the F1 score by 20% compared to the best scores of the SOTA approaches. Additionally, CompVPD contributes to security practice by helping identify 20 vulnerability patches and 18 fixes for high-risk bugs from 2,500 recent code commits in five highly popular open-source projects.
- [61] arXiv:2310.06300 (replaced) [pdf, ps, html, other]
-
Title: An Empirically Grounded Reference Architecture for Software Supply Chain Metadata ManagementComments: Accepted for full paper presentation at EASE 2024 conferenceSubjects: Cryptography and Security (cs.CR); Software Engineering (cs.SE)
With the rapid rise in Software Supply Chain (SSC) attacks, organisations need thorough and trustworthy visibility over the entire SSC of their software inventory to detect risks early and identify compromised assets rapidly in the event of an SSC attack. One way to achieve such visibility is through SSC metadata, machine-readable and authenticated documents describing an artefact's lifecycle. Adopting SSC metadata requires organisations to procure or develop a Software Supply Chain Metadata Management system (SCM2), a suite of software tools for performing life cycle activities of SSC metadata documents such as creation, signing, distribution, and consumption. Selecting or developing an SCM2 is challenging due to the lack of a comprehensive domain model and architectural blueprint to aid practitioners in navigating the vast design space of SSC metadata terminologies, frameworks, and solutions. This paper addresses the above-mentioned challenge by presenting an empirically grounded Reference Architecture (RA) comprising of a domain model and an architectural blueprint for SCM2 systems. Our proposed RA is constructed systematically on an empirical foundation built with industry-driven and peer-reviewed SSC security frameworks. Our theoretical evaluation, which consists of an architectural mapping of five prominent SSC security tools on the RA, ensures its validity and applicability, thus affirming the proposed RA as an effective framework for analysing existing SCM2 solutions and guiding the engineering of new SCM2 systems.
- [62] arXiv:2310.20448 (replaced) [pdf, ps, html, other]
-
Title: A Survey on Federated Unlearning: Challenges, Methods, and Future DirectionsSubjects: Cryptography and Security (cs.CR)
In recent years, the notion of ``the right to be forgotten" (RTBF) has become a crucial aspect of data privacy, requiring the provision of mechanisms that support the removal of personal data of individuals upon their requests. Consequently, given the extensive adoption of data-intensive machine learning (ML) algorithms and increasing concerns for personal data privacy protection, the concept of machine unlearning (MU) has gained considerable attention. MU empowers an ML model to selectively eliminate identifiable information. Evolving from the foundational principles of MU, federated unlearning (FU) has emerged to confront the challenge of data erasure within federated learning (FL) settings. This empowers the FL model to unlearn an FL client or identifiable information pertaining to the client. Nevertheless, unlike traditional MU, the distinctive attributes of federated learning introduce specific challenges for FU techniques. These challenges necessitate a tailored design when developing FU algorithms. While various concepts and numerous federated unlearning schemes exist in this field, the unified workflow and tailored design of FU are not yet well understood. Therefore, this comprehensive survey delves into the techniques, methodologies, and recent advancements in federated unlearning. It provides an overview of fundamental concepts and principles, evaluates existing federated unlearning algorithms, and reviews optimizations tailored to federated learning. Additionally, it discusses practical applications and assesses their limitations. Finally, it outlines promising directions for future research.
- [63] arXiv:2311.16169 (replaced) [pdf, ps, html, other]
-
Title: Understanding the Effectiveness of Large Language Models in Detecting Security VulnerabilitiesSubjects: Cryptography and Security (cs.CR); Programming Languages (cs.PL); Software Engineering (cs.SE)
Security vulnerabilities in modern software are prevalent and harmful. While automated vulnerability detection tools have made promising progress, their scalability and applicability remain challenging. Recently, Large Language Models (LLMs), such as GPT-4 and CodeLlama, have demonstrated remarkable performance on code-related tasks. However, it is unknown whether such LLMs can do complex reasoning over code. In this work, we explore whether pre-trained LLMs can detect security vulnerabilities and address the limitations of existing tools. We evaluate the effectiveness of pre-trained LLMs, in terms of performance, explainability, and robustness, on a set of five diverse security benchmarks spanning two languages, Java and C/C++, and covering both synthetic and real-world projects.
Overall, all LLMs show modest effectiveness in end-to-end reasoning about vulnerabilities, obtaining an average of 60% accuracy across all datasets. However, we observe that LLMs show promising abilities at performing parts of the analysis correctly, such as identifying vulnerability-related specifications (e.g., sources and sinks) and leveraging natural language information to understand code behavior (e.g., to check if code is sanitized). Further, LLMs are relatively much better at detecting simpler vulnerabilities that typically only need local reasoning (e.g., Integer Overflows and NULL pointer dereference). We find that advanced prompting strategies that involve step-by-step analysis significantly improve performance of LLMs on real-world datasets (improving F1 score by up to 0.25 on average). Finally, we share our insights and recommendations for future work on leveraging LLMs for vulnerability detection. - [64] arXiv:2312.00027 (replaced) [pdf, ps, html, other]
-
Title: Stealthy and Persistent Unalignment on Large Language Models via Backdoor InjectionsSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Recent developments in Large Language Models (LLMs) have manifested significant advancements. To facilitate safeguards against malicious exploitation, a body of research has concentrated on aligning LLMs with human preferences and inhibiting their generation of inappropriate content. Unfortunately, such alignments are often vulnerable: fine-tuning with a minimal amount of harmful data can easily unalign the target LLM. While being effective, such fine-tuning-based unalignment approaches also have their own limitations: (1) non-stealthiness, after fine-tuning, safety audits or red-teaming can easily expose the potential weaknesses of the unaligned models, thereby precluding their release/use. (2) non-persistence, the unaligned LLMs can be easily repaired through re-alignment, i.e., fine-tuning again with aligned data points. In this work, we show that it is possible to conduct stealthy and persistent unalignment on large language models via backdoor injections. We also provide a novel understanding on the relationship between the backdoor persistence and the activation pattern and further provide guidelines for potential trigger design. Through extensive experiments, we demonstrate that our proposed stealthy and persistent unalignment can successfully pass the safety evaluation while maintaining strong persistence against re-alignment defense.
- [65] arXiv:2312.13985 (replaced) [pdf, ps, html, other]
-
Title: R\'enyi Pufferfish Privacy: General Additive Noise Mechanisms and Privacy Amplification by IterationSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG); Machine Learning (stat.ML)
Pufferfish privacy is a flexible generalization of differential privacy that allows to model arbitrary secrets and adversary's prior knowledge about the data. Unfortunately, designing general and tractable Pufferfish mechanisms that do not compromise utility is challenging. Furthermore, this framework does not provide the composition guarantees needed for a direct use in iterative machine learning algorithms. To mitigate these issues, we introduce a Rényi divergence-based variant of Pufferfish and show that it allows us to extend the applicability of the Pufferfish framework. We first generalize the Wasserstein mechanism to cover a wide range of noise distributions and introduce several ways to improve its utility. We also derive stronger guarantees against out-of-distribution adversaries. Finally, as an alternative to composition, we prove privacy amplification results for contractive noisy iterations and showcase the first use of Pufferfish in private convex optimization. A common ingredient underlying our results is the use and extension of shift reduction lemmas.
- [66] arXiv:2402.08614 (replaced) [pdf, ps, html, other]
-
Title: CaPS: Collaborative and Private Synthetic Data Generation from Distributed SourcesComments: In Proceedings of the 41st International Conference on Machine Learning, 2024Subjects: Cryptography and Security (cs.CR)
Data is the lifeblood of the modern world, forming a fundamental part of AI, decision-making, and research advances. With increase in interest in data, governments have taken important steps towards a regulated data world, drastically impacting data sharing and data usability and resulting in massive amounts of data confined within the walls of organizations. While synthetic data generation (SDG) is an appealing solution to break down these walls and enable data sharing, the main drawback of existing solutions is the assumption of a trusted aggregator for generative model training. Given that many data holders may not want to, or be legally allowed to, entrust a central entity with their raw data, we propose a framework for the collaborative and private generation of synthetic tabular data from distributed data holders. Our solution is general, applicable to any marginal-based SDG, and provides input privacy by replacing the trusted aggregator with secure multi-party computation (MPC) protocols and output privacy via differential privacy (DP). We demonstrate the applicability and scalability of our approach for the state-of-the-art select-measure-generate SDG algorithms MWEM+PGM and AIM.
- [67] arXiv:2402.18104 (replaced) [pdf, ps, html, other]
-
Title: Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and ReconstructionSubjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
In recent years, large language models (LLMs) have demonstrated notable success across various tasks, but the trustworthiness of LLMs is still an open problem. One specific threat is the potential to generate toxic or harmful responses. Attackers can craft adversarial prompts that induce harmful responses from LLMs. In this work, we pioneer a theoretical foundation in LLMs security by identifying bias vulnerabilities within the safety fine-tuning and design a black-box jailbreak method named DRA (Disguise and Reconstruction Attack), which conceals harmful instructions through disguise and prompts the model to reconstruct the original harmful instruction within its completion. We evaluate DRA across various open-source and closed-source models, showcasing state-of-the-art jailbreak success rates and attack efficiency. Notably, DRA boasts a 91.1% attack success rate on OpenAI GPT-4 chatbot.
- [68] arXiv:2402.19200 (replaced) [pdf, ps, html, other]
-
Title: PRSA: PRompt Stealing Attacks against Large Language ModelsSubjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL)
In recent years, "prompt as a service" has greatly enhanced the utility of large language models (LLMs) by enabling them to perform various downstream tasks efficiently without fine-tuning. This has also increased the commercial value of prompts. However, the potential risk of leakage in these commercialized prompts remains largely underexplored. In this paper, we introduce a novel attack framework, PRSA, designed for prompt stealing attacks against LLMs. The main idea of PRSA is to infer the intent behind a prompt by analyzing its input-output content, enabling the generation of a surrogate prompt that replicates the original's functionality. Specifically, PRSA mainly consists of two key phases: prompt mutation and prompt pruning. In the mutation phase, we propose a prompt attention algorithm based on output difference. The algorithm facilitates the generation of effective surrogate prompts by learning key factors that influence the accurate inference of prompt intent. During the pruning phase, we employ a two-step related word identification strategy to detect and mask words that are highly related to the input, thus improving the generalizability of the surrogate prompts. We verify the actual threat of PRSA through evaluation in both real-world settings, non-interactive and interactive prompt services. The results strongly confirm the PRSA's effectiveness and generalizability. We have reported these findings to prompt service providers and actively collaborate with them to implement defensive measures.
- [69] arXiv:2403.01472 (replaced) [pdf, ps, html, other]
-
Title: WARDEN: Multi-Directional Backdoor Watermarks for Embedding-as-a-Service Copyright ProtectionComments: Accepted to ACL2024 (Main Proceedings)Subjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG)
Embedding as a Service (EaaS) has become a widely adopted solution, which offers feature extraction capabilities for addressing various downstream tasks in Natural Language Processing (NLP). Prior studies have shown that EaaS can be prone to model extraction attacks; nevertheless, this concern could be mitigated by adding backdoor watermarks to the text embeddings and subsequently verifying the attack models post-publication. Through the analysis of the recent watermarking strategy for EaaS, EmbMarker, we design a novel CSE (Clustering, Selection, Elimination) attack that removes the backdoor watermark while maintaining the high utility of embeddings, indicating that the previous watermarking approach can be breached. In response to this new threat, we propose a new protocol to make the removal of watermarks more challenging by incorporating multiple possible watermark directions. Our defense approach, WARDEN, notably increases the stealthiness of watermarks and has been empirically shown to be effective against CSE attack.
- [70] arXiv:2405.08479 (replaced) [pdf, ps, html, other]
-
Title: A Survey on Complexity Measures of Pseudo-Random SequencesSubjects: Cryptography and Security (cs.CR)
Since the introduction of the Kolmogorov complexity of binary sequences in the 1960s, there have been significant advancements in the topic of complexity measures for randomness assessment, which are of fundamental importance in theoretical computer science and of practical interest in cryptography. This survey reviews notable research from the past four decades on the linear, quadratic and maximum-order complexities of pseudo-random sequences and their relations with Lempel-Ziv complexity, expansion complexity, 2-adic complexity, and correlation measures.
- [71] arXiv:2405.12786 (replaced) [pdf, ps, html, other]
-
Title: Rethinking the Vulnerabilities of Face Recognition Systems:From a Practical PerspectiveJiahao Chen, Zhiqiang Shen, Yuwen Pu, Chunyi Zhou, Changjiang Li, Jiliang Li, Ting Wang, Shouling JiComments: 19 pages,version 3Subjects: Cryptography and Security (cs.CR)
Face Recognition Systems (FRS) have increasingly integrated into critical applications, including surveillance and user authentication, highlighting their pivotal role in modern security systems. Recent studies have revealed vulnerabilities in FRS to adversarial (e.g., adversarial patch attacks) and backdoor attacks (e.g., training data poisoning), raising significant concerns about their reliability and trustworthiness. Previous studies primarily focus on traditional adversarial or backdoor attacks, overlooking the resource-intensive or privileged-manipulation nature of such threats, thus limiting their practical generalization, stealthiness, universality and robustness. Correspondingly, in this paper, we delve into the inherent vulnerabilities in FRS through user studies and preliminary explorations. By exploiting these vulnerabilities, we identify a novel attack, facial identity backdoor attack dubbed FIBA, which unveils a potentially more devastating threat against FRS:an enrollment-stage backdoor attack. FIBA circumvents the limitations of traditional attacks, enabling broad-scale disruption by allowing any attacker donning a specific trigger to bypass these systems. This implies that after a single, poisoned example is inserted into the database, the corresponding trigger becomes a universal key for any attackers to spoof the FRS. This strategy essentially challenges the conventional attacks by initiating at the enrollment stage, dramatically transforming the threat landscape by poisoning the feature database rather than the training data.
- [72] arXiv:2405.13147 (replaced) [pdf, ps, html, other]
-
Title: A novel reliability attack of Physical Unclonable FunctionsSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
Physical Unclonable Functions (PUFs) are emerging as promising security primitives for IoT devices, providing device fingerprints based on physical characteristics. Despite their strengths, PUFs are vulnerable to machine learning (ML) attacks, including conventional and reliability-based attacks. Conventional ML attacks have been effective in revealing vulnerabilities of many PUFs, and reliability-based ML attacks are more powerful tools that have detected vulnerabilities of some PUFs that are resistant to conventional ML attacks. Since reliability-based ML attacks leverage information of PUFs' unreliability, we were tempted to examine the feasibility of building defense using reliability enhancing techniques, and have discovered that majority voting with reasonably high repeats provides effective defense against existing reliability-based ML attack methods. It is known that majority voting reduces but does not eliminate unreliability, we are motivated to investigate if new attack methods exist that can capture the low unreliability of highly but not-perfectly reliable PUFs, which led to the development of a new reliability representation and the new representation-enabled attack method that has experimentally cracked PUFs enhanced with majority voting of high repetitions.
- [73] arXiv:2405.13804 (replaced) [pdf, ps, html, other]
-
Title: Guarding Multiple Secrets: Enhanced Summary Statistic Privacy for Data SharingSubjects: Cryptography and Security (cs.CR)
Data sharing enables critical advances in many research areas and business applications, but it may lead to inadvertent disclosure of sensitive summary statistics (e.g., means or quantiles). Existing literature only focuses on protecting a single confidential quantity, while in practice, data sharing involves multiple sensitive statistics. We propose a novel framework to define, analyze, and protect multi-secret summary statistics privacy in data sharing. Specifically, we measure the privacy risk of any data release mechanism by the worst-case probability of an attacker successfully inferring summary statistic secrets. Given an attacker's objective spanning from inferring a subset to the entirety of summary statistic secrets, we systematically design and analyze tailored privacy metrics. Defining the distortion as the worst-case distance between the original and released data distribution, we analyze the tradeoff between privacy and distortion. Our contribution also includes designing and analyzing data release mechanisms tailored for different data distributions and secret types. Evaluations on real-world data demonstrate the effectiveness of our mechanisms in practical applications.
- [74] arXiv:2405.18255 (replaced) [pdf, ps, html, other]
-
Title: Channel Reciprocity Based Attack Detection for Securing UWB Ranging by AutoencoderSubjects: Cryptography and Security (cs.CR); Social and Information Networks (cs.SI); Signal Processing (eess.SP)
A variety of ranging threats represented by Ghost Peak attack have raised concerns regarding the security performance of Ultra-Wide Band (UWB) systems with the finalization of the IEEE 802.15.4z standard. Based on channel reciprocity, this paper proposes a low complexity attack detection scheme that compares Channel Impulse Response (CIR) features of both ranging sides utilizing an autoencoder with the capability of data compression and feature extraction. Taking Ghost Peak attack as an example, this paper demonstrates the effectiveness, feasibility and generalizability of the proposed attack detection scheme through simulation and experimental validation. The proposed scheme achieves an attack detection success rate of over 99% and can be implemented in current systems at low cost.
- [75] arXiv:2405.19971 (replaced) [pdf, ps, html, other]
-
Title: GasTrace: Detecting Sandwich Attack Malicious Accounts in EthereumSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
The openness and transparency of Ethereum transaction data make it easy to be exploited by any entities, executing malicious attacks. The sandwich attack manipulates the Automated Market Maker (AMM) mechanism, profiting from manipulating the market price through front or after-running transactions. To identify and prevent sandwich attacks, we propose a cascade classification framework GasTrace. GasTrace analyzes various transaction features to detect malicious accounts, notably through the analysis and modeling of Gas features. In the initial classification, we utilize the Support Vector Machine (SVM) with the Radial Basis Function (RBF) kernel to generate the predicted probabilities of accounts, further constructing a detailed transaction network. Subsequently, the behavior features are captured by the Graph Attention Network (GAT) technique in the second classification. Through cascade classification, GasTrace can analyze and classify the sandwich attacks. Our experimental results demonstrate that GasTrace achieves a remarkable detection and generation capability, performing an accuracy of 96.73% and an F1 score of 95.71% for identifying sandwich attack accounts.
- [76] arXiv:2406.00799 (replaced) [pdf, ps, html, other]
-
Title: Are you still on track!? Catching LLM Task Drift with ActivationsSubjects: Cryptography and Security (cs.CR); Computation and Language (cs.CL); Computers and Society (cs.CY)
Large Language Models (LLMs) are routinely used in retrieval-augmented applications to orchestrate tasks and process inputs from users and other sources. These inputs, even in a single LLM interaction, can come from a variety of sources, of varying trustworthiness and provenance. This opens the door to prompt injection attacks, where the LLM receives and acts upon instructions from supposedly data-only sources, thus deviating from the user's original instructions. We define this as task drift, and we propose to catch it by scanning and analyzing the LLM's activations. We compare the LLM's activations before and after processing the external input in order to detect whether this input caused instruction drift. We develop two probing methods and find that simply using a linear classifier can detect drift with near perfect ROC AUC on an out-of-distribution test set. We show that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions, without being trained on any of these attacks. Our setup does not require any modification of the LLM (e.g., fine-tuning) or any text generation, thus maximizing deployability and cost efficiency and avoiding reliance on unreliable model output. To foster future research on activation-based task inspection, decoding, and interpretability, we will release our large-scale TaskTracker toolkit, comprising a dataset of over 500K instances, representations from 4 SoTA language models, and inspection tools.
- [77] arXiv:2406.03230 (replaced) [pdf, ps, html, other]
-
Title: Defending Large Language Models Against Attacks With Residual Stream Activation AnalysisSubjects: Cryptography and Security (cs.CR); Machine Learning (cs.LG)
The widespread adoption of Large Language Models (LLMs), exemplified by OpenAI's ChatGPT, brings to the forefront the imperative to defend against adversarial threats on these models. These attacks, which manipulate an LLM's output by introducing malicious inputs, undermine the model's integrity and the trust users place in its outputs. In response to this challenge, our paper presents an innovative defensive strategy, given white box access to an LLM, that harnesses residual activation analysis between transformer layers of the LLM. We apply a novel methodology for analyzing distinctive activation patterns in the residual streams for attack prompt classification. We curate multiple datasets to demonstrate how this method of classification has high accuracy across multiple types of attack scenarios, including our newly-created attack dataset. Furthermore, we enhance the model's resilience by integrating safety fine-tuning techniques for LLMs in order to measure its effect on our capability to detect attacks. The results underscore the effectiveness of our approach in enhancing the detection and mitigation of adversarial inputs, advancing the security framework within which LLMs operate.
- [78] arXiv:2312.16818 (replaced) [pdf, ps, html, other]
-
Title: Challenges in Drone Firmware Analyses of Drone Firmware and Its SolutionsSubjects: Robotics (cs.RO); Cryptography and Security (cs.CR)
With the advancement of Internet of Things (IoT) technology, its applications span various sectors such as public, industrial, private and military. In particular, the drone sector has gained significant attention for both commercial and military purposes. As a result, there has been a surge in research focused on vulnerability analysis of drones. However, most security research to mitigate threats to IoT devices has focused primarily on networks, firmware and mobile applications. Of these, the use of fuzzing to analyze the security of firmware requires emulation of the firmware. However, when it comes to drone firmware, the industry lacks emulation and automated fuzzing tools. This is largely due to challenges such as limited input interfaces, firmware encryption and signatures. While it may be tempting to assume that existing emulators and automated analyzers for IoT devices can be applied to drones, practical applications have proven otherwise. In this paper, we discuss the challenges of dynamically analyzing drone firmware and propose potential solutions. In addition, we demonstrate the effectiveness of our methodology by applying it to DJI drones, which have the largest market share.
- [79] arXiv:2402.06255 (replaced) [pdf, ps, html, other]
-
Title: Fight Back Against Jailbreaking via Prompt Adversarial TuningSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Cryptography and Security (cs.CR)
While Large Language Models (LLMs) have achieved tremendous success in various applications, they are also susceptible to jailbreak attacks. Several primary defense strategies have been proposed to protect LLMs from producing harmful information, mostly with a particular focus on harmful content filtering or heuristical defensive prompt designs. However, how to achieve intrinsic robustness through the prompts remains an open problem. In this paper, motivated by adversarial training paradigms for achieving reliable robustness, we propose an approach named Prompt Adversarial Tuning (PAT) that trains a prompt control attached to the user prompt as a guard prefix. To achieve our defense goal whilst maintaining natural performance, we optimize the control prompt with both adversarial and benign prompts. Comprehensive experiments show that our method is effective against both black-box and white-box attacks, reducing the success rate of advanced attacks to nearly 0 while maintaining the model's utility on the benign task. The proposed defense strategy incurs only negligible computational overhead, charting a new perspective for future explorations in LLM security. Our code is available at this https URL.
- [80] arXiv:2402.11399 (replaced) [pdf, ps, html, other]
-
Title: k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated TextComments: Accepted to ACL 24 FindingsSubjects: Computation and Language (cs.CL); Cryptography and Security (cs.CR); Computers and Society (cs.CY); Machine Learning (cs.LG)
Recent watermarked generation algorithms inject detectable signatures during language generation to facilitate post-hoc detection. While token-level watermarks are vulnerable to paraphrase attacks, SemStamp (Hou et al., 2023) applies watermark on the semantic representation of sentences and demonstrates promising robustness. SemStamp employs locality-sensitive hashing (LSH) to partition the semantic space with arbitrary hyperplanes, which results in a suboptimal tradeoff between robustness and speed. We propose k-SemStamp, a simple yet effective enhancement of SemStamp, utilizing k-means clustering as an alternative of LSH to partition the embedding space with awareness of inherent semantic structure. Experimental results indicate that k-SemStamp saliently improves its robustness and sampling efficiency while preserving the generation quality, advancing a more effective tool for machine-generated text detection.
- [81] arXiv:2402.11989 (replaced) [pdf, ps, html, other]
-
Title: Privacy-Preserving Low-Rank Adaptation for Latent Diffusion ModelsSubjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
Low-rank adaptation (LoRA) is an efficient strategy for adapting latent diffusion models (LDMs) on a private dataset to generate specific images by minimizing the adaptation loss. However, the LoRA-adapted LDMs are vulnerable to membership inference (MI) attacks that can judge whether a particular data point belongs to the private dataset, thus leading to the privacy leakage. To defend against MI attacks, we first propose a straightforward solution: Membership-Privacy-preserving LoRA (MP-LoRA). MP-LoRA is formulated as a min-max optimization problem where a proxy attack model is trained by maximizing its MI gain while the LDM is adapted by minimizing the sum of the adaptation loss and the MI gain of the proxy attack model. However, we empirically find that MP-LoRA has the issue of unstable optimization, and theoretically analyze that the potential reason is the unconstrained local smoothness, which impedes the privacy-preserving adaptation. To mitigate this issue, we further propose a Stable Membership-Privacy-preserving LoRA (SMP-LoRA) that adapts the LDM by minimizing the ratio of the adaptation loss to the MI gain. Besides, we theoretically prove that the local smoothness of SMP-LoRA can be constrained by the gradient norm, leading to improved convergence. Our experimental results corroborate that SMP-LoRA can indeed defend against MI attacks and generate high-quality images. Our code is available at this https URL.
- [82] arXiv:2403.07865 (replaced) [pdf, ps, html, other]
-
Title: CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code CompletionComments: ACL Findings 2024, Code is available at this https URLSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Machine Learning (cs.LG); Software Engineering (cs.SE)
The rapid advancement of Large Language Models (LLMs) has brought about remarkable generative capabilities but also raised concerns about their potential misuse. While strategies like supervised fine-tuning and reinforcement learning from human feedback have enhanced their safety, these methods primarily focus on natural languages, which may not generalize to other domains. This paper introduces CodeAttack, a framework that transforms natural language inputs into code inputs, presenting a novel environment for testing the safety generalization of LLMs. Our comprehensive studies on state-of-the-art LLMs including GPT-4, Claude-2, and Llama-2 series reveal a new and universal safety vulnerability of these models against code input: CodeAttack bypasses the safety guardrails of all models more than 80\% of the time. We find that a larger distribution gap between CodeAttack and natural language leads to weaker safety generalization, such as encoding natural language input with data structures. Furthermore, we give our hypotheses about the success of CodeAttack: the misaligned bias acquired by LLMs during code training, prioritizing code completion over avoiding the potential safety risk. Finally, we analyze potential mitigation measures. These findings highlight new safety risks in the code domain and the need for more robust safety alignment algorithms to match the code capabilities of LLMs.
- [83] arXiv:2403.17787 (replaced) [pdf, ps, html, other]
-
Title: Evaluating the Efficacy of Prompt-Engineered Large Multimodal Models Versus Fine-Tuned Vision Transformers in Image-Based Security ApplicationsSubjects: Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR); Computer Vision and Pattern Recognition (cs.CV)
The success of Large Language Models (LLMs) has led to a parallel rise in the development of Large Multimodal Models (LMMs), which have begun to transform a variety of applications. These sophisticated multimodal models are designed to interpret and analyze complex data by integrating multiple modalities such as text and images, thereby opening new avenues for a range of applications. This paper investigates the applicability and effectiveness of prompt-engineered LMMs that process both images and text, including models such as LLaVA, BakLLaVA, Moondream, Gemini-pro-vision, and GPT-4o, compared to fine-tuned Vision Transformer (ViT) models in addressing critical security challenges. We focus on two distinct security tasks: 1) a visually evident task of detecting simple triggers, such as small pixel variations in images that could be exploited to access potential backdoors in the models, and 2) a visually non-evident task of malware classification through visual representations. In the visually evident task, some LMMs, such as Gemini-pro-vision and GPT-4o, have demonstrated the potential to achieve good performance with careful prompt engineering, with GPT-4o achieving the highest accuracy and F1-score of 91.9\% and 91\%, respectively. However, the fine-tuned ViT models exhibit perfect performance in this task due to its simplicity. For the visually non-evident task, the results highlight a significant divergence in performance, with ViT models achieving F1-scores of 97.11\% in predicting 25 malware classes and 97.61\% in predicting 5 malware families, whereas LMMs showed suboptimal performance despite iterative prompt improvements. This study not only showcases the strengths and limitations of prompt-engineered LMMs in cybersecurity applications but also emphasizes the unmatched efficacy of fine-tuned ViT models for precise and dependable tasks.