A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Filters
Do Pre-trained Language Models Indeed Understand Software Engineering Tasks?
[article]
2022
arXiv
pre-print
Preserved Fulltext
Artificial intelligence (AI) for software engineering (SE) tasks has recently achieved promising performance. ...
In this paper, we investigate to what extent the pre-trained language model truly understands those SE tasks such as code search, code summarization, etc. ...
GPT, BERT, XLNet: Masking & SIS Overinterpretation is not dependent on software engineering tasks and is also prevalent in pretrained language models. What is the impact of overinterpretation? ...
arXiv:2211.10623v1
fatcat:5krqodh73jbu7gegfzrfjpwbt4
Automating Code-Related Tasks Through Transformers: The Impact of Pre-training
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
Transformers have gained popularity in the software engineering (SE) literature. ...
Indeed, MLM is just one of the possible pre-training objectives and recent work from the natural language processing field suggest that pre-training objectives tailored for the specific downstream task ...
Transformers achieved state-of-the-art performance in several natural language processing (NLP) tasks and, recently, have gained popularity in the software engineering (SE) literature for the automation ...
arXiv:2302.04048v1
fatcat:srwscgkkqveuxedy3vxvlsmk7m
Linguistically-Informed Neural Architectures for Lexical, Syntactic and Semantic Tasks in Sanskrit
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
The first task, Sanskrit Word Segmentation (SWS), is a fundamental text processing task for any other downstream applications. ...
The primary focus of this thesis is to make Sanskrit manuscripts more accessible to the end-users through natural language technologies. ...
The DCST variant of our pretraining approach is also the best among its peers, although the performance of Oracle MI model in this case is indeed the upper bound. ...
arXiv:2308.08807v1
fatcat:teab3mj6frddvaycs3whkvnvrm
DexBERT: Effective, Task-Agnostic and Fine-grained Representation Learning of Android Bytecode
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We empirically assess whether DexBERT is able to model the DEX language and evaluate the suitability of our model in three distinct class-level software engineering tasks: Malicious Code Localization, ...
The automation of a large number of software engineering tasks is becoming possible thanks to Machine Learning (ML). ...
Universal models can also be useful in the field of software engineering, where many tasks that take software artifacts as input are investigated. ...
arXiv:2212.05976v2
fatcat:luwgyrgdynfnvor4kucnyr55a4
UniLog: Deploy One Model and Specialize it for All Log Analysis Tasks
[article]
2021
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
UniLog: Deploy One Model and Specialize it for All Log Analysis Tasks ...
Task Dataset Method Precision Recall F1
Prefix Language Model 0.65 0.68 0.66 Pretrain Dataset Task Precision Recall F1 Comp. ...
In 2020 IEEE 31st International Symposium on
Software Reliability Engineering (ISSRE), pages 37–47. IEEE, 2020. ...
arXiv:2112.03159v1
fatcat:zqiycpha7vd3xmpey4loxbrd3a
Using Transfer Learning for Code-Related Tasks
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Then, these models are fine-tuned to support specific tasks of interest (e.g, language translation). ...
In particular, pre-trained transformer models are on the rise, also thanks to the excellent results they achieved in Natural Language Processing (NLP) tasks. ...
Any opinions, findings, and conclusions expressed herein are the authors' and do not necessarily reflect those of the sponsors. ...
arXiv:2206.08574v1
fatcat:jszf3dckjfejdjndyxn3xkqviq
On Measuring Social Biases in Prompt-Based Multi-Task Learning
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Large language models trained on a mixture of NLP tasks that are converted into a text-to-text format using prompts, can generalize into novel forms of language and handle novel tasks. ...
In this paper, we study T0, a large-scale multi-task text-to-text language model trained using prompt-based learning. ...
Related Works In order to obtain a strong task-specific model to tackle various NLP tasks, the de facto practice has been to use a pretrained language model and fine-tune it on a downstream task (Alberti ...
arXiv:2205.11605v1
fatcat:ofcldcog7bcedo4voonzh4zeme
LINSPECTOR: Multilingual Probing Tasks for Word Representations
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
Other Versions
We then present experiments on several diverse multilingual word embedding models, in which we relate the probing task performance for a diverse set of languages to a range of five classic NLP tasks: POS-tagging ...
Despite an ever growing number of word representation models introduced for a large number of languages, there is a lack of a standardized technique to provide insights into what is captured by these models ...
We would like to thank Marvin Kaster for his help on contextualizing the probing tasks and to Max Eichler for his contribution on acquiring experimental results for additional languages in Appendix L. ...
arXiv:1903.09442v2
fatcat:eq5cgujvsfh43p23rewzpt6m34
Multilingual Probing Tasks for Word Representations
2020
Computational Linguistics
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2021; you can also visit the original URL.
The file type is application/pdf.
We then present experiments on several diverse multilingual word embedding models, in which we relate the probing task performance for a diverse set of languages to a range of five classic NLP tasks: POS-tagging ...
Despite an ever growing number of word representation models introduced for a large number of languages, there is a lack of a standardized technique to provide insights into what is captured by these models ...
We would like to thank Marvin Kaster for his help on contextualizing the probing tasks and to Max Eichler for his contribution on acquiring experimental results for additional languages in Appendix C. ...
doi:10.1162/coli_a_00376
fatcat:kmbeuimsxbf6fflau2lv4vufv4
A Comprehensive Study of Vision Transformers in Image Classification Tasks
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2023 pre-print arXiv:2312.01232v1 fatcat:2nhp3dtukjafdfzjbcx4ibhxdq
Then, we present Vision Transformers models in chronological order, starting with early attempts at adapting attention mechanism to vision tasks followed by the adoption of vision transformers, as they ...
Image Classification is a fundamental task in the field of computer vision that frequently serves as a benchmark for gauging advancements in Computer Vision. ...
He is currently a software engineer at PointClickCare inc. ...
arXiv:2312.01232v2
fatcat:sg3ulxriebdtzc4cvei34di3s4
PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
The growing dependence on Large Language Models (LLMs) for finishing user instructions necessitates a comprehensive understanding of their robustness to complex task completion in real-world situations ...
To assess the robustness of Language Models to software versions, we vary the number of provided APIs to simulate both the newest version and earlier version settings. ...
Introduction Large Language Models (e.g. ...
arXiv:2403.03788v1
fatcat:yuqwgebulvdvbohojvhntj4efq
Is ChatGPT a General-Purpose Natural Language Processing Task Solver?
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
Note that this fulltext copy is not of the "primary" version of this work. The version it corresponds to is:
2023 pre-print arXiv:2302.06476v2 fatcat:ptnu2rqtnfai3hh3z4plzjycdmOther Versions
Spurred by advancements in scale, large language models (LLMs) have demonstrated the ability to perform a variety of natural language processing (NLP) tasks zero-shot -- i.e., without adaptation on downstream ...
However, it is not yet known whether ChatGPT can serve as a generalist model that can perform many NLP tasks zero-shot. ...
Natural Language Inference Table 4 : Accuracy (%) of different models on natural language inference tasks (RTE and CB). ...
arXiv:2302.06476v3
fatcat:ujla2xtp4ndfdmxopvi2zprvyi
Pretrained AI Models: Performativity, Mobility, and Change
[article]
2019
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2020; you can also visit the original URL.
The file type is application/pdf.
We discuss how pretrained models are developed and compared under the common task framework, but that this may make self-regulation inadequate. ...
We close by discussing how this sociological understanding of pretrained models can inform AI governance frameworks for fairness, accountability, and transparency. ...
Pursuing Holy Grails In building engineering systems-whether physical systems like engines or informational ones like AI-benchmarking performance to understand how well one is doing is often cast as important ...
arXiv:1909.03290v1
fatcat:7doni7tc3rginpokkow2wtiqmy
FETA: Towards Specializing Foundation Models for Expert Task Applications
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
language descriptions. ...
In this paper, we propose a first of its kind FETA benchmark built around the task of teaching FMs to understand technical documentation, via learning to match their graphical illustrations to corresponding ...
The goal of these works is to learn foundational language and vision representations that are required for language and vision understanding. ...
arXiv:2209.03648v2
fatcat:xzne446owbdufatjl2rhv36lvq
CoditT5: Pretraining for Source Code and Natural Language Editing
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
To address this, we propose a novel pretraining objective which explicitly models edits and use it to build CoditT5, a large language model for software-related editing tasks that is pretrained on large ...
Pretrained language models have been shown to be effective in many software-related generation tasks; however, they are not well-suited for editing tasks as they are not designed to reason about edits. ...
pretrained models for software engineering. ...
arXiv:2208.05446v2
fatcat:nivresaxv5d2zoaumxatww6vyq
« Previous
Showing results 1 — 15 out of 1,856 results